Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/schnet_feat_single_feats_no_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_single_feats_no_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=1),
 'dropout': 0.4,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': True,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_single/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 400,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 400,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_single_feats_no_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_single/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/schnet_feat_single/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 281,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.822866
Epoch 1
Validation binary_cross_entropy = 3.515413
Epoch 2
Validation binary_cross_entropy = 2.339975
Epoch 3
Validation binary_cross_entropy = 0.287842
Epoch 4
Loss = 3.7690e+00, PNorm = 27.3556, GNorm = 53.4136, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.306978
Epoch 5
Validation binary_cross_entropy = 0.495309
Epoch 6
Validation binary_cross_entropy = 1.561908
Epoch 7
Validation binary_cross_entropy = 0.575925
Epoch 8
Validation binary_cross_entropy = 0.345531
Epoch 9
Loss = 3.8031e+00, PNorm = 27.3701, GNorm = 49.2261, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.299570
Epoch 10
Validation binary_cross_entropy = 0.484001
Epoch 11
Validation binary_cross_entropy = 1.710573
Epoch 12
Validation binary_cross_entropy = 1.085004
Epoch 13
Validation binary_cross_entropy = 0.600870
Epoch 14
Loss = 2.3858e+00, PNorm = 27.3970, GNorm = 33.1306, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.442640
Epoch 15
Validation binary_cross_entropy = 0.567543
Epoch 16
Validation binary_cross_entropy = 0.721508
Epoch 17
Validation binary_cross_entropy = 0.878399
Epoch 18
Validation binary_cross_entropy = 0.806271
Epoch 19
Loss = 1.4309e+00, PNorm = 27.4217, GNorm = 20.3412, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.809989
Epoch 20
Validation binary_cross_entropy = 0.800853
Epoch 21
Validation binary_cross_entropy = 0.954849
Epoch 22
Validation binary_cross_entropy = 0.711131
Epoch 23
Validation binary_cross_entropy = 0.580195
Epoch 24
Loss = 1.4231e+00, PNorm = 27.4404, GNorm = 24.2422, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.583087
Epoch 25
Validation binary_cross_entropy = 0.825678
Epoch 26
Validation binary_cross_entropy = 1.021997
Epoch 27
Validation binary_cross_entropy = 0.735271
Epoch 28
Validation binary_cross_entropy = 0.566802
Epoch 29
Loss = 1.1698e+00, PNorm = 27.4550, GNorm = 18.7520, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.471172
Epoch 30
Validation binary_cross_entropy = 0.406079
Epoch 31
Validation binary_cross_entropy = 0.464958
Epoch 32
Validation binary_cross_entropy = 0.780962
Epoch 33
Validation binary_cross_entropy = 1.313637
Epoch 34
Loss = 1.2111e+00, PNorm = 27.4681, GNorm = 21.7704, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 1.695014
Epoch 35
Validation binary_cross_entropy = 1.220629
Epoch 36
Validation binary_cross_entropy = 0.551533
Epoch 37
Validation binary_cross_entropy = 0.400614
Epoch 38
Validation binary_cross_entropy = 0.396099
Epoch 39
Loss = 1.3000e+00, PNorm = 27.4793, GNorm = 21.2527, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.561625
Epoch 40
Validation binary_cross_entropy = 0.816296
Epoch 41
Validation binary_cross_entropy = 0.765827
Epoch 42
Validation binary_cross_entropy = 0.613432
Epoch 43
Validation binary_cross_entropy = 0.474468
Epoch 44
Loss = 1.1771e+00, PNorm = 27.4913, GNorm = 23.6461, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.463159
Epoch 45
Validation binary_cross_entropy = 0.529993
Epoch 46
Validation binary_cross_entropy = 0.585166
Epoch 47
Validation binary_cross_entropy = 0.694581
Epoch 48
Validation binary_cross_entropy = 0.642014
Epoch 49
Loss = 1.1039e+00, PNorm = 27.5030, GNorm = 19.2985, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.566131
Epoch 50
Validation binary_cross_entropy = 0.538317
Epoch 51
Validation binary_cross_entropy = 0.539877
Epoch 52
Validation binary_cross_entropy = 0.537833
Epoch 53
Validation binary_cross_entropy = 0.567615
Epoch 54
Loss = 9.0386e-01, PNorm = 27.5121, GNorm = 10.8694, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.529158
Epoch 55
Validation binary_cross_entropy = 0.491456
Epoch 56
Validation binary_cross_entropy = 0.486035
Epoch 57
Validation binary_cross_entropy = 0.529763
Epoch 58
Validation binary_cross_entropy = 0.585303
Epoch 59
Loss = 7.9569e-01, PNorm = 27.5192, GNorm = 11.4062, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.635812
Epoch 60
Validation binary_cross_entropy = 0.623294
Epoch 61
Validation binary_cross_entropy = 0.570702
Epoch 62
Validation binary_cross_entropy = 0.540572
Epoch 63
Validation binary_cross_entropy = 0.498925
Epoch 64
Loss = 9.2615e-01, PNorm = 27.5268, GNorm = 15.2583, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.517457
Epoch 65
Validation binary_cross_entropy = 0.588789
Epoch 66
Validation binary_cross_entropy = 0.669145
Epoch 67
Validation binary_cross_entropy = 0.704705
Epoch 68
Validation binary_cross_entropy = 0.685849
Epoch 69
Loss = 7.0213e-01, PNorm = 27.5325, GNorm = 10.1672, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.662677
Epoch 70
Validation binary_cross_entropy = 0.613516
Epoch 71
Validation binary_cross_entropy = 0.555828
Epoch 72
Validation binary_cross_entropy = 0.542565
Epoch 73
Validation binary_cross_entropy = 0.531924
Epoch 74
Loss = 6.5168e-01, PNorm = 27.5377, GNorm = 9.3446, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.533353
Epoch 75
Validation binary_cross_entropy = 0.567388
Epoch 76
Validation binary_cross_entropy = 0.610624
Epoch 77
Validation binary_cross_entropy = 0.670910
Epoch 78
Validation binary_cross_entropy = 0.710891
Epoch 79
Loss = 8.4486e-01, PNorm = 27.5439, GNorm = 11.2564, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.669602
Epoch 80
Validation binary_cross_entropy = 0.615012
Epoch 81
Validation binary_cross_entropy = 0.583630
Epoch 82
Validation binary_cross_entropy = 0.557648
Epoch 83
Validation binary_cross_entropy = 0.554992
Epoch 84
Loss = 8.3216e-01, PNorm = 27.5503, GNorm = 11.0920, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.533609
Epoch 85
Validation binary_cross_entropy = 0.528310
Epoch 86
Validation binary_cross_entropy = 0.515217
Epoch 87
Validation binary_cross_entropy = 0.535435
Epoch 88
Validation binary_cross_entropy = 0.559010
Epoch 89
Loss = 7.7800e-01, PNorm = 27.5546, GNorm = 7.0860, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.597906
Epoch 90
Validation binary_cross_entropy = 0.636842
Epoch 91
Validation binary_cross_entropy = 0.625085
Epoch 92
Validation binary_cross_entropy = 0.589947
Epoch 93
Validation binary_cross_entropy = 0.564707
Epoch 94
Loss = 6.5527e-01, PNorm = 27.5589, GNorm = 6.9571, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.577577
Epoch 95
Validation binary_cross_entropy = 0.583429
Epoch 96
Validation binary_cross_entropy = 0.580124
Epoch 97
Validation binary_cross_entropy = 0.580594
Epoch 98
Validation binary_cross_entropy = 0.594698
Epoch 99
Loss = 5.8011e-01, PNorm = 27.5641, GNorm = 8.2470, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.596656
Epoch 100
Validation binary_cross_entropy = 0.608718
Epoch 101
Validation binary_cross_entropy = 0.631102
Epoch 102
Validation binary_cross_entropy = 0.636931
Epoch 103
Validation binary_cross_entropy = 0.618035
Epoch 104
Loss = 5.5125e-01, PNorm = 27.5705, GNorm = 6.5624, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.610573
Epoch 105
Validation binary_cross_entropy = 0.608292
Epoch 106
Validation binary_cross_entropy = 0.611566
Epoch 107
Validation binary_cross_entropy = 0.625385
Epoch 108
Validation binary_cross_entropy = 0.621016
Epoch 109
Loss = 6.5204e-01, PNorm = 27.5762, GNorm = 5.7160, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.623224
Epoch 110
Validation binary_cross_entropy = 0.630392
Epoch 111
Validation binary_cross_entropy = 0.633462
Epoch 112
Validation binary_cross_entropy = 0.654567
Epoch 113
Validation binary_cross_entropy = 0.659227
Epoch 114
Loss = 6.2585e-01, PNorm = 27.5813, GNorm = 8.0454, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.630004
Epoch 115
Validation binary_cross_entropy = 0.615375
Epoch 116
Validation binary_cross_entropy = 0.614175
Epoch 117
Validation binary_cross_entropy = 0.616988
Epoch 118
Validation binary_cross_entropy = 0.648166
Epoch 119
Loss = 5.7075e-01, PNorm = 27.5854, GNorm = 5.5101, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.672040
Epoch 120
Validation binary_cross_entropy = 0.673594
Epoch 121
Validation binary_cross_entropy = 0.631066
Epoch 122
Validation binary_cross_entropy = 0.605781
Epoch 123
Validation binary_cross_entropy = 0.604074
Epoch 124
Loss = 5.9728e-01, PNorm = 27.5912, GNorm = 7.9061, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.600216
Epoch 125
Validation binary_cross_entropy = 0.604443
Epoch 126
Validation binary_cross_entropy = 0.620338
Epoch 127
Validation binary_cross_entropy = 0.621362
Epoch 128
Validation binary_cross_entropy = 0.614431
Epoch 129
Loss = 5.0495e-01, PNorm = 27.5952, GNorm = 6.9729, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.612285
Epoch 130
Validation binary_cross_entropy = 0.614106
Epoch 131
Validation binary_cross_entropy = 0.608925
Epoch 132
Validation binary_cross_entropy = 0.593923
Epoch 133
Validation binary_cross_entropy = 0.582679
Epoch 134
Loss = 4.8953e-01, PNorm = 27.6005, GNorm = 5.5579, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.592878
Epoch 135
Validation binary_cross_entropy = 0.603273
Epoch 136
Validation binary_cross_entropy = 0.598888
Epoch 137
Validation binary_cross_entropy = 0.588729
Epoch 138
Validation binary_cross_entropy = 0.576846
Epoch 139
Loss = 6.5975e-01, PNorm = 27.6050, GNorm = 8.5253, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.563084
Epoch 140
Validation binary_cross_entropy = 0.556802
Epoch 141
Validation binary_cross_entropy = 0.555598
Epoch 142
Validation binary_cross_entropy = 0.553398
Epoch 143
Validation binary_cross_entropy = 0.547210
Epoch 144
Loss = 5.4821e-01, PNorm = 27.6087, GNorm = 4.4556, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.531239
Epoch 145
Validation binary_cross_entropy = 0.523443
Epoch 146
Validation binary_cross_entropy = 0.520917
Epoch 147
Validation binary_cross_entropy = 0.521592
Epoch 148
Validation binary_cross_entropy = 0.528877
Epoch 149
Loss = 5.4383e-01, PNorm = 27.6121, GNorm = 7.3338, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.540074
Epoch 150
Validation binary_cross_entropy = 0.540103
Epoch 151
Validation binary_cross_entropy = 0.537305
Epoch 152
Validation binary_cross_entropy = 0.532137
Epoch 153
Validation binary_cross_entropy = 0.517100
Epoch 154
Loss = 5.9709e-01, PNorm = 27.6156, GNorm = 7.5188, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.504610
Epoch 155
Validation binary_cross_entropy = 0.491246
Epoch 156
Validation binary_cross_entropy = 0.494404
Epoch 157
Validation binary_cross_entropy = 0.502072
Epoch 158
Validation binary_cross_entropy = 0.516045
Epoch 159
Loss = 5.1804e-01, PNorm = 27.6202, GNorm = 5.3813, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.530850
Epoch 160
Validation binary_cross_entropy = 0.550321
Epoch 161
Validation binary_cross_entropy = 0.568745
Epoch 162
Validation binary_cross_entropy = 0.579745
Epoch 163
Validation binary_cross_entropy = 0.581617
Epoch 164
Loss = 4.9328e-01, PNorm = 27.6243, GNorm = 5.9289, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.582317
Epoch 165
Validation binary_cross_entropy = 0.576890
Epoch 166
Validation binary_cross_entropy = 0.567563
Epoch 167
Validation binary_cross_entropy = 0.559382
Epoch 168
Validation binary_cross_entropy = 0.550893
Epoch 169
Loss = 5.6073e-01, PNorm = 27.6277, GNorm = 6.9712, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.546429
Epoch 170
Validation binary_cross_entropy = 0.556240
Epoch 171
Validation binary_cross_entropy = 0.557050
Epoch 172
Validation binary_cross_entropy = 0.552699
Epoch 173
Validation binary_cross_entropy = 0.546608
Epoch 174
Loss = 5.4901e-01, PNorm = 27.6314, GNorm = 5.6385, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.538552
Epoch 175
Validation binary_cross_entropy = 0.541458
Epoch 176
Validation binary_cross_entropy = 0.543433
Epoch 177
Validation binary_cross_entropy = 0.551216
Epoch 178
Validation binary_cross_entropy = 0.558675
Epoch 179
Loss = 4.9005e-01, PNorm = 27.6361, GNorm = 6.2359, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.561759
Epoch 180
Validation binary_cross_entropy = 0.578325
Epoch 181
Validation binary_cross_entropy = 0.595730
Epoch 182
Validation binary_cross_entropy = 0.608869
Epoch 183
Validation binary_cross_entropy = 0.616677
Epoch 184
Loss = 4.7389e-01, PNorm = 27.6404, GNorm = 4.8818, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.635081
Epoch 185
Validation binary_cross_entropy = 0.633631
Epoch 186
Validation binary_cross_entropy = 0.614706
Epoch 187
Validation binary_cross_entropy = 0.601013
Epoch 188
Validation binary_cross_entropy = 0.588502
Epoch 189
Loss = 5.8405e-01, PNorm = 27.6439, GNorm = 5.4924, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.582088
Epoch 190
Validation binary_cross_entropy = 0.567902
Epoch 191
Validation binary_cross_entropy = 0.564619
Epoch 192
Validation binary_cross_entropy = 0.548644
Epoch 193
Validation binary_cross_entropy = 0.530901
Epoch 194
Loss = 4.3211e-01, PNorm = 27.6473, GNorm = 6.4033, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.518560
Epoch 195
Validation binary_cross_entropy = 0.512949
Epoch 196
Validation binary_cross_entropy = 0.517146
Epoch 197
Validation binary_cross_entropy = 0.521322
Epoch 198
Validation binary_cross_entropy = 0.535187
Epoch 199
Loss = 4.5727e-01, PNorm = 27.6507, GNorm = 5.7959, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.551339
Epoch 200
Validation binary_cross_entropy = 0.569028
Epoch 201
Validation binary_cross_entropy = 0.589944
Epoch 202
Validation binary_cross_entropy = 0.609557
Epoch 203
Validation binary_cross_entropy = 0.623109
Epoch 204
Loss = 4.6970e-01, PNorm = 27.6541, GNorm = 5.7594, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.623123
Epoch 205
Validation binary_cross_entropy = 0.615406
Epoch 206
Validation binary_cross_entropy = 0.602104
Epoch 207
Validation binary_cross_entropy = 0.587373
Epoch 208
Validation binary_cross_entropy = 0.574963
Epoch 209
Loss = 4.0911e-01, PNorm = 27.6587, GNorm = 4.4998, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.563017
Epoch 210
Validation binary_cross_entropy = 0.566841
Epoch 211
Validation binary_cross_entropy = 0.574710
Epoch 212
Validation binary_cross_entropy = 0.580448
Epoch 213
Validation binary_cross_entropy = 0.589758
Epoch 214
Loss = 5.6779e-01, PNorm = 27.6629, GNorm = 4.2684, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.592574
Epoch 215
Validation binary_cross_entropy = 0.597740
Epoch 216
Validation binary_cross_entropy = 0.601777
Epoch 217
Validation binary_cross_entropy = 0.604243
Epoch 218
Validation binary_cross_entropy = 0.605400
Epoch 219
Loss = 5.1722e-01, PNorm = 27.6681, GNorm = 5.2604, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.606794
Epoch 220
Validation binary_cross_entropy = 0.612136
Epoch 221
Validation binary_cross_entropy = 0.617169
Epoch 222
Validation binary_cross_entropy = 0.611240
Epoch 223
Validation binary_cross_entropy = 0.601229
Epoch 224
Loss = 4.7639e-01, PNorm = 27.6723, GNorm = 8.3676, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.594567
Epoch 225
Validation binary_cross_entropy = 0.590887
Epoch 226
Validation binary_cross_entropy = 0.583263
Epoch 227
Validation binary_cross_entropy = 0.575304
Epoch 228
Validation binary_cross_entropy = 0.572073
Epoch 229
Loss = 5.4881e-01, PNorm = 27.6766, GNorm = 5.5360, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.570377
Epoch 230
Validation binary_cross_entropy = 0.566809
Epoch 231
Validation binary_cross_entropy = 0.564521
Epoch 232
Validation binary_cross_entropy = 0.569572
Epoch 233
Validation binary_cross_entropy = 0.569747
Epoch 234
Loss = 3.8743e-01, PNorm = 27.6816, GNorm = 5.0685, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.571581
Epoch 235
Validation binary_cross_entropy = 0.574635
Epoch 236
Validation binary_cross_entropy = 0.570346
Epoch 237
Validation binary_cross_entropy = 0.563589
Epoch 238
Validation binary_cross_entropy = 0.562220
Epoch 239
Loss = 4.0919e-01, PNorm = 27.6858, GNorm = 9.2887, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.562678
Epoch 240
Validation binary_cross_entropy = 0.562213
Epoch 241
Validation binary_cross_entropy = 0.560420
Epoch 242
Validation binary_cross_entropy = 0.559732
Epoch 243
Validation binary_cross_entropy = 0.561877
Epoch 244
Loss = 5.0834e-01, PNorm = 27.6897, GNorm = 5.8934, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.565453
Epoch 245
Validation binary_cross_entropy = 0.570632
Epoch 246
Validation binary_cross_entropy = 0.572635
Epoch 247
Validation binary_cross_entropy = 0.575951
Epoch 248
Validation binary_cross_entropy = 0.576583
Epoch 249
Loss = 6.3707e-01, PNorm = 27.6931, GNorm = 8.6499, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.575305
Epoch 250
Validation binary_cross_entropy = 0.575938
Epoch 251
Validation binary_cross_entropy = 0.580152
Epoch 252
Validation binary_cross_entropy = 0.586372
Epoch 253
Validation binary_cross_entropy = 0.591418
Epoch 254
Loss = 4.8707e-01, PNorm = 27.6967, GNorm = 4.2367, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.590897
Epoch 255
Validation binary_cross_entropy = 0.586132
Epoch 256
Validation binary_cross_entropy = 0.572754
Epoch 257
Validation binary_cross_entropy = 0.560076
Epoch 258
Validation binary_cross_entropy = 0.554389
Epoch 259
Loss = 5.2020e-01, PNorm = 27.7008, GNorm = 4.8714, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.550144
Epoch 260
Validation binary_cross_entropy = 0.544065
Epoch 261
Validation binary_cross_entropy = 0.533968
Epoch 262
Validation binary_cross_entropy = 0.528377
Epoch 263
Validation binary_cross_entropy = 0.526802
Epoch 264
Loss = 5.9235e-01, PNorm = 27.7049, GNorm = 6.5528, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.523340
Epoch 265
Validation binary_cross_entropy = 0.521132
Epoch 266
Validation binary_cross_entropy = 0.525289
Epoch 267
Validation binary_cross_entropy = 0.528836
Epoch 268
Validation binary_cross_entropy = 0.530350
Epoch 269
Loss = 4.6272e-01, PNorm = 27.7089, GNorm = 5.9409, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.532460
Epoch 270
Validation binary_cross_entropy = 0.541735
Epoch 271
Validation binary_cross_entropy = 0.555529
Epoch 272
Validation binary_cross_entropy = 0.571153
Epoch 273
Validation binary_cross_entropy = 0.582062
Epoch 274
Loss = 4.9467e-01, PNorm = 27.7133, GNorm = 4.9833, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.584836
Epoch 275
Validation binary_cross_entropy = 0.584422
Epoch 276
Validation binary_cross_entropy = 0.574933
Epoch 277
Validation binary_cross_entropy = 0.561254
Epoch 278
Validation binary_cross_entropy = 0.545222
Epoch 279
Loss = 4.8920e-01, PNorm = 27.7177, GNorm = 3.9104, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.536912
Epoch 280
Validation binary_cross_entropy = 0.529519
Epoch 281
Validation binary_cross_entropy = 0.529745
Epoch 282
Validation binary_cross_entropy = 0.534599
Epoch 283
Validation binary_cross_entropy = 0.538803
Epoch 284
Loss = 5.5590e-01, PNorm = 27.7224, GNorm = 4.9488, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.541384
Epoch 285
Validation binary_cross_entropy = 0.546369
Epoch 286
Validation binary_cross_entropy = 0.554926
Epoch 287
Validation binary_cross_entropy = 0.561509
Epoch 288
Validation binary_cross_entropy = 0.555205
Epoch 289
Loss = 4.7625e-01, PNorm = 27.7261, GNorm = 5.8775, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.552161
Epoch 290
Validation binary_cross_entropy = 0.548520
Epoch 291
Validation binary_cross_entropy = 0.539567
Epoch 292
Validation binary_cross_entropy = 0.532849
Epoch 293
Validation binary_cross_entropy = 0.524085
Epoch 294
Loss = 4.7838e-01, PNorm = 27.7307, GNorm = 4.8863, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.513051
Epoch 295
Validation binary_cross_entropy = 0.506435
Epoch 296
Validation binary_cross_entropy = 0.503962
Epoch 297
Validation binary_cross_entropy = 0.509133
Epoch 298
Validation binary_cross_entropy = 0.521431
Epoch 299
Loss = 5.3932e-01, PNorm = 27.7344, GNorm = 4.5922, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.536563
Model 0 best validation binary_cross_entropy = 0.287842 on epoch 3
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.286525
Ensemble test binary_cross_entropy = 0.286525
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 281,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.015762
Epoch 1
Validation binary_cross_entropy = 2.164739
Epoch 2
Validation binary_cross_entropy = 0.868970
Epoch 3
Validation binary_cross_entropy = 0.346177
Epoch 4
Loss = 3.6054e+00, PNorm = 27.3557, GNorm = 52.7278, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.456225
Epoch 5
Validation binary_cross_entropy = 0.414871
Epoch 6
Validation binary_cross_entropy = 1.107769
Epoch 7
Validation binary_cross_entropy = 1.015564
Epoch 8
Validation binary_cross_entropy = 0.431918
Epoch 9
Loss = 2.8563e+00, PNorm = 27.3729, GNorm = 35.3254, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.407944
Epoch 10
Validation binary_cross_entropy = 0.744340
Epoch 11
Validation binary_cross_entropy = 1.317900
Epoch 12
Validation binary_cross_entropy = 1.607225
Epoch 13
Validation binary_cross_entropy = 1.425103
Epoch 14
Loss = 1.7405e+00, PNorm = 27.4011, GNorm = 15.9020, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 1.023384
Epoch 15
Validation binary_cross_entropy = 0.864527
Epoch 16
Validation binary_cross_entropy = 0.906698
Epoch 17
Validation binary_cross_entropy = 0.724224
Epoch 18
Validation binary_cross_entropy = 0.623830
Epoch 19
Loss = 1.9736e+00, PNorm = 27.4242, GNorm = 27.4389, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.832542
Epoch 20
Validation binary_cross_entropy = 1.267997
Epoch 21
Validation binary_cross_entropy = 1.119877
Epoch 22
Validation binary_cross_entropy = 0.437229
Epoch 23
Validation binary_cross_entropy = 0.375149
Epoch 24
Loss = 1.8725e+00, PNorm = 27.4439, GNorm = 29.4152, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.364424
Epoch 25
Validation binary_cross_entropy = 0.426494
Epoch 26
Validation binary_cross_entropy = 0.779961
Epoch 27
Validation binary_cross_entropy = 1.020696
Epoch 28
Validation binary_cross_entropy = 1.139379
Epoch 29
Loss = 1.4649e+00, PNorm = 27.4600, GNorm = 20.6432, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.705788
Epoch 30
Validation binary_cross_entropy = 0.538827
Epoch 31
Validation binary_cross_entropy = 0.581056
Epoch 32
Validation binary_cross_entropy = 0.907448
Epoch 33
Validation binary_cross_entropy = 1.315060
Epoch 34
Loss = 1.3110e+00, PNorm = 27.4738, GNorm = 16.6667, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 1.262116
Epoch 35
Validation binary_cross_entropy = 0.920009
Epoch 36
Validation binary_cross_entropy = 0.554130
Epoch 37
Validation binary_cross_entropy = 0.431191
Epoch 38
Validation binary_cross_entropy = 0.459053
Epoch 39
Loss = 1.7069e+00, PNorm = 27.4847, GNorm = 18.5108, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.562895
Epoch 40
Validation binary_cross_entropy = 0.705946
Epoch 41
Validation binary_cross_entropy = 0.724220
Epoch 42
Validation binary_cross_entropy = 0.628495
Epoch 43
Validation binary_cross_entropy = 0.608894
Epoch 44
Loss = 1.0103e+00, PNorm = 27.4952, GNorm = 11.2870, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.614323
Epoch 45
Validation binary_cross_entropy = 0.647638
Epoch 46
Validation binary_cross_entropy = 0.698055
Epoch 47
Validation binary_cross_entropy = 0.701526
Epoch 48
Validation binary_cross_entropy = 0.598933
Epoch 49
Loss = 8.1691e-01, PNorm = 27.5039, GNorm = 12.9779, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.468845
Epoch 50
Validation binary_cross_entropy = 0.465318
Epoch 51
Validation binary_cross_entropy = 0.546391
Epoch 52
Validation binary_cross_entropy = 0.708140
Epoch 53
Validation binary_cross_entropy = 0.813037
Epoch 54
Loss = 8.9183e-01, PNorm = 27.5119, GNorm = 14.3115, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.861208
Epoch 55
Validation binary_cross_entropy = 0.838430
Epoch 56
Validation binary_cross_entropy = 0.819658
Epoch 57
Validation binary_cross_entropy = 0.700385
Epoch 58
Validation binary_cross_entropy = 0.566839
Epoch 59
Loss = 8.2105e-01, PNorm = 27.5205, GNorm = 10.9090, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.483636
Epoch 60
Validation binary_cross_entropy = 0.455722
Epoch 61
Validation binary_cross_entropy = 0.491167
Epoch 62
Validation binary_cross_entropy = 0.515559
Epoch 63
Validation binary_cross_entropy = 0.522096
Epoch 64
Loss = 1.0846e+00, PNorm = 27.5286, GNorm = 15.1981, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.512483
Epoch 65
Validation binary_cross_entropy = 0.446793
Epoch 66
Validation binary_cross_entropy = 0.395808
Epoch 67
Validation binary_cross_entropy = 0.386427
Epoch 68
Validation binary_cross_entropy = 0.400273
Epoch 69
Loss = 1.0830e+00, PNorm = 27.5347, GNorm = 12.1846, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.431192
Epoch 70
Validation binary_cross_entropy = 0.455471
Epoch 71
Validation binary_cross_entropy = 0.468872
Epoch 72
Validation binary_cross_entropy = 0.486252
Epoch 73
Validation binary_cross_entropy = 0.507221
Epoch 74
Loss = 7.3208e-01, PNorm = 27.5408, GNorm = 8.6402, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.514179
Epoch 75
Validation binary_cross_entropy = 0.516263
Epoch 76
Validation binary_cross_entropy = 0.523814
Epoch 77
Validation binary_cross_entropy = 0.542797
Epoch 78
Validation binary_cross_entropy = 0.573301
Epoch 79
Loss = 8.9164e-01, PNorm = 27.5468, GNorm = 11.6787, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.595216
Epoch 80
Validation binary_cross_entropy = 0.623422
Epoch 81
Validation binary_cross_entropy = 0.654798
Epoch 82
Validation binary_cross_entropy = 0.663740
Epoch 83
Validation binary_cross_entropy = 0.638137
Epoch 84
Loss = 6.6639e-01, PNorm = 27.5522, GNorm = 11.8305, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.605577
Epoch 85
Validation binary_cross_entropy = 0.554595
Epoch 86
Validation binary_cross_entropy = 0.523366
Epoch 87
Validation binary_cross_entropy = 0.528939
Epoch 88
Validation binary_cross_entropy = 0.582297
Epoch 89
Loss = 6.4590e-01, PNorm = 27.5578, GNorm = 7.3858, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.640339
Epoch 90
Validation binary_cross_entropy = 0.713357
Epoch 91
Validation binary_cross_entropy = 0.797922
Epoch 92
Validation binary_cross_entropy = 0.865707
Epoch 93
Validation binary_cross_entropy = 0.866670
Epoch 94
Loss = 6.5328e-01, PNorm = 27.5627, GNorm = 9.1941, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.805482
Epoch 95
Validation binary_cross_entropy = 0.729415
Epoch 96
Validation binary_cross_entropy = 0.657799
Epoch 97
Validation binary_cross_entropy = 0.549669
Epoch 98
Validation binary_cross_entropy = 0.478019
Epoch 99
Loss = 5.8568e-01, PNorm = 27.5673, GNorm = 9.4593, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.442802
Epoch 100
Validation binary_cross_entropy = 0.458874
Epoch 101
Validation binary_cross_entropy = 0.496135
Epoch 102
Validation binary_cross_entropy = 0.570903
Epoch 103
Validation binary_cross_entropy = 0.623109
Epoch 104
Loss = 6.6338e-01, PNorm = 27.5723, GNorm = 7.4680, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.648122
Epoch 105
Validation binary_cross_entropy = 0.643589
Epoch 106
Validation binary_cross_entropy = 0.634238
Epoch 107
Validation binary_cross_entropy = 0.602573
Epoch 108
Validation binary_cross_entropy = 0.576015
Epoch 109
Loss = 6.0984e-01, PNorm = 27.5779, GNorm = 9.2083, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.568459
Epoch 110
Validation binary_cross_entropy = 0.554372
Epoch 111
Validation binary_cross_entropy = 0.541654
Epoch 112
Validation binary_cross_entropy = 0.515678
Epoch 113
Validation binary_cross_entropy = 0.494005
Epoch 114
Loss = 7.2803e-01, PNorm = 27.5823, GNorm = 8.6124, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.491884
Epoch 115
Validation binary_cross_entropy = 0.508175
Epoch 116
Validation binary_cross_entropy = 0.524957
Epoch 117
Validation binary_cross_entropy = 0.535879
Epoch 118
Validation binary_cross_entropy = 0.525100
Epoch 119
Loss = 8.3343e-01, PNorm = 27.5871, GNorm = 8.3698, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.506746
Epoch 120
Validation binary_cross_entropy = 0.508244
Epoch 121
Validation binary_cross_entropy = 0.497317
Epoch 122
Validation binary_cross_entropy = 0.497927
Epoch 123
Validation binary_cross_entropy = 0.500074
Epoch 124
Loss = 6.7524e-01, PNorm = 27.5918, GNorm = 7.6411, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.514395
Epoch 125
Validation binary_cross_entropy = 0.535544
Epoch 126
Validation binary_cross_entropy = 0.551760
Epoch 127
Validation binary_cross_entropy = 0.566059
Epoch 128
Validation binary_cross_entropy = 0.562899
Epoch 129
Loss = 5.1984e-01, PNorm = 27.5967, GNorm = 9.3275, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.552775
Epoch 130
Validation binary_cross_entropy = 0.544479
Epoch 131
Validation binary_cross_entropy = 0.542950
Epoch 132
Validation binary_cross_entropy = 0.543940
Epoch 133
Validation binary_cross_entropy = 0.548288
Epoch 134
Loss = 5.9160e-01, PNorm = 27.6017, GNorm = 6.2704, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.560622
Epoch 135
Validation binary_cross_entropy = 0.565140
Epoch 136
Validation binary_cross_entropy = 0.545216
Epoch 137
Validation binary_cross_entropy = 0.532617
Epoch 138
Validation binary_cross_entropy = 0.544986
Epoch 139
Loss = 6.0008e-01, PNorm = 27.6073, GNorm = 5.4420, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.574135
Epoch 140
Validation binary_cross_entropy = 0.595714
Epoch 141
Validation binary_cross_entropy = 0.606140
Epoch 142
Validation binary_cross_entropy = 0.612516
Epoch 143
Validation binary_cross_entropy = 0.607402
Epoch 144
Loss = 5.5501e-01, PNorm = 27.6128, GNorm = 7.8141, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.588228
Epoch 145
Validation binary_cross_entropy = 0.560645
Epoch 146
Validation binary_cross_entropy = 0.541721
Epoch 147
Validation binary_cross_entropy = 0.519753
Epoch 148
Validation binary_cross_entropy = 0.512405
Epoch 149
Loss = 5.5733e-01, PNorm = 27.6171, GNorm = 6.7079, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.505701
Epoch 150
Validation binary_cross_entropy = 0.508927
Epoch 151
Validation binary_cross_entropy = 0.513608
Epoch 152
Validation binary_cross_entropy = 0.521984
Epoch 153
Validation binary_cross_entropy = 0.536744
Epoch 154
Loss = 5.3893e-01, PNorm = 27.6211, GNorm = 6.2681, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.552031
Epoch 155
Validation binary_cross_entropy = 0.555395
Epoch 156
Validation binary_cross_entropy = 0.551160
Epoch 157
Validation binary_cross_entropy = 0.542143
Epoch 158
Validation binary_cross_entropy = 0.546185
Epoch 159
Loss = 5.0746e-01, PNorm = 27.6250, GNorm = 6.4991, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.559658
Epoch 160
Validation binary_cross_entropy = 0.578253
Epoch 161
Validation binary_cross_entropy = 0.606221
Epoch 162
Validation binary_cross_entropy = 0.630645
Epoch 163
Validation binary_cross_entropy = 0.641871
Epoch 164
Loss = 6.1174e-01, PNorm = 27.6294, GNorm = 7.8352, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.628134
Epoch 165
Validation binary_cross_entropy = 0.593582
Epoch 166
Validation binary_cross_entropy = 0.555042
Epoch 167
Validation binary_cross_entropy = 0.524919
Epoch 168
Validation binary_cross_entropy = 0.518105
Epoch 169
Loss = 5.6049e-01, PNorm = 27.6334, GNorm = 5.8739, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.524225
Epoch 170
Validation binary_cross_entropy = 0.530224
Epoch 171
Validation binary_cross_entropy = 0.547661
Epoch 172
Validation binary_cross_entropy = 0.573688
Epoch 173
Validation binary_cross_entropy = 0.595161
Epoch 174
Loss = 6.2361e-01, PNorm = 27.6381, GNorm = 7.4716, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.604769
Epoch 175
Validation binary_cross_entropy = 0.599876
Epoch 176
Validation binary_cross_entropy = 0.578934
Epoch 177
Validation binary_cross_entropy = 0.558077
Epoch 178
Validation binary_cross_entropy = 0.540061
Epoch 179
Loss = 4.5805e-01, PNorm = 27.6423, GNorm = 6.4970, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.523513
Epoch 180
Validation binary_cross_entropy = 0.517431
Epoch 181
Validation binary_cross_entropy = 0.514540
Epoch 182
Validation binary_cross_entropy = 0.519151
Epoch 183
Validation binary_cross_entropy = 0.526676
Epoch 184
Loss = 5.7090e-01, PNorm = 27.6456, GNorm = 9.1944, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.535421
Epoch 185
Validation binary_cross_entropy = 0.539260
Epoch 186
Validation binary_cross_entropy = 0.539639
Epoch 187
Validation binary_cross_entropy = 0.547213
Epoch 188
Validation binary_cross_entropy = 0.559917
Epoch 189
Loss = 6.2729e-01, PNorm = 27.6494, GNorm = 7.7264, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.573295
Epoch 190
Validation binary_cross_entropy = 0.592813
Epoch 191
Validation binary_cross_entropy = 0.603179
Epoch 192
Validation binary_cross_entropy = 0.594016
Epoch 193
Validation binary_cross_entropy = 0.580895
Epoch 194
Loss = 5.2277e-01, PNorm = 27.6527, GNorm = 9.0158, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.580132
Epoch 195
Validation binary_cross_entropy = 0.582420
Epoch 196
Validation binary_cross_entropy = 0.572789
Epoch 197
Validation binary_cross_entropy = 0.569809
Epoch 198
Validation binary_cross_entropy = 0.568492
Epoch 199
Loss = 4.6316e-01, PNorm = 27.6559, GNorm = 7.0487, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.578312
Epoch 200
Validation binary_cross_entropy = 0.600413
Epoch 201
Validation binary_cross_entropy = 0.613948
Epoch 202
Validation binary_cross_entropy = 0.617626
Epoch 203
Validation binary_cross_entropy = 0.614718
Epoch 204
Loss = 5.7599e-01, PNorm = 27.6593, GNorm = 4.6509, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.600756
Epoch 205
Validation binary_cross_entropy = 0.570039
Epoch 206
Validation binary_cross_entropy = 0.553024
Epoch 207
Validation binary_cross_entropy = 0.546147
Epoch 208
Validation binary_cross_entropy = 0.548012
Epoch 209
Loss = 4.2479e-01, PNorm = 27.6630, GNorm = 3.7568, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.550762
Epoch 210
Validation binary_cross_entropy = 0.569206
Epoch 211
Validation binary_cross_entropy = 0.580113
Epoch 212
Validation binary_cross_entropy = 0.578983
Epoch 213
Validation binary_cross_entropy = 0.584370
Epoch 214
Loss = 4.7239e-01, PNorm = 27.6664, GNorm = 5.2174, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.594916
Epoch 215
Validation binary_cross_entropy = 0.601422
Epoch 216
Validation binary_cross_entropy = 0.597184
Epoch 217
Validation binary_cross_entropy = 0.580050
Epoch 218
Validation binary_cross_entropy = 0.571503
Epoch 219
Loss = 6.0443e-01, PNorm = 27.6703, GNorm = 6.9963, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.569202
Epoch 220
Validation binary_cross_entropy = 0.566468
Epoch 221
Validation binary_cross_entropy = 0.559091
Epoch 222
Validation binary_cross_entropy = 0.549232
Epoch 223
Validation binary_cross_entropy = 0.542675
Epoch 224
Loss = 6.3913e-01, PNorm = 27.6733, GNorm = 8.1792, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.542315
Epoch 225
Validation binary_cross_entropy = 0.538517
Epoch 226
Validation binary_cross_entropy = 0.539586
Epoch 227
Validation binary_cross_entropy = 0.543125
Epoch 228
Validation binary_cross_entropy = 0.551626
Epoch 229
Loss = 4.7233e-01, PNorm = 27.6756, GNorm = 4.8737, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.563460
Epoch 230
Validation binary_cross_entropy = 0.578470
Epoch 231
Validation binary_cross_entropy = 0.595110
Epoch 232
Validation binary_cross_entropy = 0.608321
Epoch 233
Validation binary_cross_entropy = 0.617002
Epoch 234
Loss = 4.8588e-01, PNorm = 27.6784, GNorm = 6.0214, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.620488
Epoch 235
Validation binary_cross_entropy = 0.612662
Epoch 236
Validation binary_cross_entropy = 0.603118
Epoch 237
Validation binary_cross_entropy = 0.590717
Epoch 238
Validation binary_cross_entropy = 0.574905
Epoch 239
Loss = 4.8296e-01, PNorm = 27.6822, GNorm = 5.1067, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.562998
Epoch 240
Validation binary_cross_entropy = 0.562315
Epoch 241
Validation binary_cross_entropy = 0.562667
Epoch 242
Validation binary_cross_entropy = 0.561359
Epoch 243
Validation binary_cross_entropy = 0.562234
Epoch 244
Loss = 5.6264e-01, PNorm = 27.6861, GNorm = 7.7245, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.560505
Epoch 245
Validation binary_cross_entropy = 0.561157
Epoch 246
Validation binary_cross_entropy = 0.564578
Epoch 247
Validation binary_cross_entropy = 0.571114
Epoch 248
Validation binary_cross_entropy = 0.578913
Epoch 249
Loss = 5.2222e-01, PNorm = 27.6899, GNorm = 5.9732, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.581236
Epoch 250
Validation binary_cross_entropy = 0.575908
Epoch 251
Validation binary_cross_entropy = 0.569587
Epoch 252
Validation binary_cross_entropy = 0.562675
Epoch 253
Validation binary_cross_entropy = 0.557393
Epoch 254
Loss = 5.9607e-01, PNorm = 27.6944, GNorm = 7.0261, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.556021
Epoch 255
Validation binary_cross_entropy = 0.561168
Epoch 256
Validation binary_cross_entropy = 0.565533
Epoch 257
Validation binary_cross_entropy = 0.566277
Epoch 258
Validation binary_cross_entropy = 0.561772
Epoch 259
Loss = 5.1077e-01, PNorm = 27.6984, GNorm = 6.6463, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.561665
Epoch 260
Validation binary_cross_entropy = 0.570890
Epoch 261
Validation binary_cross_entropy = 0.584342
Epoch 262
Validation binary_cross_entropy = 0.595414
Epoch 263
Validation binary_cross_entropy = 0.604355
Epoch 264
Loss = 2.9468e-01, PNorm = 27.7022, GNorm = 2.7616, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.612262
Epoch 265
Validation binary_cross_entropy = 0.613407
Epoch 266
Validation binary_cross_entropy = 0.609735
Epoch 267
Validation binary_cross_entropy = 0.603270
Epoch 268
Validation binary_cross_entropy = 0.598850
Epoch 269
Loss = 3.6444e-01, PNorm = 27.7057, GNorm = 3.3286, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.594552
Epoch 270
Validation binary_cross_entropy = 0.591053
Epoch 271
Validation binary_cross_entropy = 0.591236
Epoch 272
Validation binary_cross_entropy = 0.594859
Epoch 273
Validation binary_cross_entropy = 0.598056
Epoch 274
Loss = 3.9816e-01, PNorm = 27.7100, GNorm = 4.2611, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.601431
Epoch 275
Validation binary_cross_entropy = 0.601906
Epoch 276
Validation binary_cross_entropy = 0.596554
Epoch 277
Validation binary_cross_entropy = 0.588244
Epoch 278
Validation binary_cross_entropy = 0.581463
Epoch 279
Loss = 4.9192e-01, PNorm = 27.7147, GNorm = 4.8485, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.582121
Epoch 280
Validation binary_cross_entropy = 0.583876
Epoch 281
Validation binary_cross_entropy = 0.585892
Epoch 282
Validation binary_cross_entropy = 0.593629
Epoch 283
Validation binary_cross_entropy = 0.599924
Epoch 284
Loss = 4.7828e-01, PNorm = 27.7181, GNorm = 2.9626, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.604294
Epoch 285
Validation binary_cross_entropy = 0.604991
Epoch 286
Validation binary_cross_entropy = 0.603854
Epoch 287
Validation binary_cross_entropy = 0.601085
Epoch 288
Validation binary_cross_entropy = 0.596385
Epoch 289
Loss = 3.6229e-01, PNorm = 27.7220, GNorm = 3.3628, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.590724
Epoch 290
Validation binary_cross_entropy = 0.589337
Epoch 291
Validation binary_cross_entropy = 0.591011
Epoch 292
Validation binary_cross_entropy = 0.586934
Epoch 293
Validation binary_cross_entropy = 0.584138
Epoch 294
Loss = 5.3213e-01, PNorm = 27.7265, GNorm = 3.7558, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.581746
Epoch 295
Validation binary_cross_entropy = 0.577792
Epoch 296
Validation binary_cross_entropy = 0.578349
Epoch 297
Validation binary_cross_entropy = 0.584158
Epoch 298
Validation binary_cross_entropy = 0.591495
Epoch 299
Loss = 4.5515e-01, PNorm = 27.7310, GNorm = 5.4901, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.600272
Model 0 best validation binary_cross_entropy = 0.346177 on epoch 3
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.363082
Ensemble test binary_cross_entropy = 0.363082
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 281,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.627318
Epoch 1
Validation binary_cross_entropy = 1.008838
Epoch 2
Validation binary_cross_entropy = 0.884907
Epoch 3
Validation binary_cross_entropy = 0.531452
Epoch 4
Loss = 3.6135e+00, PNorm = 27.3554, GNorm = 37.3925, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.338497
Epoch 5
Validation binary_cross_entropy = 0.407037
Epoch 6
Validation binary_cross_entropy = 1.033616
Epoch 7
Validation binary_cross_entropy = 1.238377
Epoch 8
Validation binary_cross_entropy = 0.935480
Epoch 9
Loss = 2.8998e+00, PNorm = 27.3723, GNorm = 36.0029, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.741732
Epoch 10
Validation binary_cross_entropy = 0.552922
Epoch 11
Validation binary_cross_entropy = 0.570759
Epoch 12
Validation binary_cross_entropy = 0.898930
Epoch 13
Validation binary_cross_entropy = 1.779298
Epoch 14
Loss = 1.5982e+00, PNorm = 27.3985, GNorm = 31.7717, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 1.723463
Epoch 15
Validation binary_cross_entropy = 1.504472
Epoch 16
Validation binary_cross_entropy = 1.070634
Epoch 17
Validation binary_cross_entropy = 0.545450
Epoch 18
Validation binary_cross_entropy = 0.450763
Epoch 19
Loss = 2.3823e+00, PNorm = 27.4203, GNorm = 18.4966, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.656392
Epoch 20
Validation binary_cross_entropy = 0.853021
Epoch 21
Validation binary_cross_entropy = 0.923214
Epoch 22
Validation binary_cross_entropy = 0.649714
Epoch 23
Validation binary_cross_entropy = 0.561372
Epoch 24
Loss = 1.6328e+00, PNorm = 27.4377, GNorm = 23.6816, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.590530
Epoch 25
Validation binary_cross_entropy = 0.669521
Epoch 26
Validation binary_cross_entropy = 0.619846
Epoch 27
Validation binary_cross_entropy = 0.569920
Epoch 28
Validation binary_cross_entropy = 0.481995
Epoch 29
Loss = 1.7163e+00, PNorm = 27.4513, GNorm = 21.9150, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.430668
Epoch 30
Validation binary_cross_entropy = 0.507509
Epoch 31
Validation binary_cross_entropy = 0.677358
Epoch 32
Validation binary_cross_entropy = 0.785722
Epoch 33
Validation binary_cross_entropy = 0.806645
Epoch 34
Loss = 1.2367e+00, PNorm = 27.4646, GNorm = 22.2896, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.773040
Epoch 35
Validation binary_cross_entropy = 0.649384
Epoch 36
Validation binary_cross_entropy = 0.511392
Epoch 37
Validation binary_cross_entropy = 0.436508
Epoch 38
Validation binary_cross_entropy = 0.443494
Epoch 39
Loss = 1.3834e+00, PNorm = 27.4772, GNorm = 14.7624, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.512542
Epoch 40
Validation binary_cross_entropy = 0.655803
Epoch 41
Validation binary_cross_entropy = 0.777430
Epoch 42
Validation binary_cross_entropy = 0.746877
Epoch 43
Validation binary_cross_entropy = 0.643304
Epoch 44
Loss = 1.1107e+00, PNorm = 27.4872, GNorm = 16.2760, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.487101
Epoch 45
Validation binary_cross_entropy = 0.443514
Epoch 46
Validation binary_cross_entropy = 0.487314
Epoch 47
Validation binary_cross_entropy = 0.533051
Epoch 48
Validation binary_cross_entropy = 0.499268
Epoch 49
Loss = 1.0595e+00, PNorm = 27.4963, GNorm = 14.4190, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.511536
Epoch 50
Validation binary_cross_entropy = 0.593706
Epoch 51
Validation binary_cross_entropy = 0.658593
Epoch 52
Validation binary_cross_entropy = 0.659991
Epoch 53
Validation binary_cross_entropy = 0.634888
Epoch 54
Loss = 1.0397e+00, PNorm = 27.5057, GNorm = 14.0995, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.599755
Epoch 55
Validation binary_cross_entropy = 0.634817
Epoch 56
Validation binary_cross_entropy = 0.663968
Epoch 57
Validation binary_cross_entropy = 0.740060
Epoch 58
Validation binary_cross_entropy = 0.759770
Epoch 59
Loss = 8.8731e-01, PNorm = 27.5138, GNorm = 13.2754, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.668911
Epoch 60
Validation binary_cross_entropy = 0.485438
Epoch 61
Validation binary_cross_entropy = 0.434900
Epoch 62
Validation binary_cross_entropy = 0.475258
Epoch 63
Validation binary_cross_entropy = 0.596341
Epoch 64
Loss = 1.2696e+00, PNorm = 27.5218, GNorm = 13.6498, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.725991
Epoch 65
Validation binary_cross_entropy = 0.877465
Epoch 66
Validation binary_cross_entropy = 0.960255
Epoch 67
Validation binary_cross_entropy = 0.802389
Epoch 68
Validation binary_cross_entropy = 0.539701
Epoch 69
Loss = 6.4171e-01, PNorm = 27.5303, GNorm = 8.1533, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.418248
Epoch 70
Validation binary_cross_entropy = 0.382493
Epoch 71
Validation binary_cross_entropy = 0.409475
Epoch 72
Validation binary_cross_entropy = 0.507892
Epoch 73
Validation binary_cross_entropy = 0.678228
Epoch 74
Loss = 5.9286e-01, PNorm = 27.5400, GNorm = 6.9576, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.862698
Epoch 75
Validation binary_cross_entropy = 0.920494
Epoch 76
Validation binary_cross_entropy = 0.829161
Epoch 77
Validation binary_cross_entropy = 0.713576
Epoch 78
Validation binary_cross_entropy = 0.613419
Epoch 79
Loss = 9.3653e-01, PNorm = 27.5479, GNorm = 11.8732, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.529167
Epoch 80
Validation binary_cross_entropy = 0.496624
Epoch 81
Validation binary_cross_entropy = 0.501323
Epoch 82
Validation binary_cross_entropy = 0.558725
Epoch 83
Validation binary_cross_entropy = 0.594672
Epoch 84
Loss = 7.9446e-01, PNorm = 27.5540, GNorm = 9.3407, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.617276
Epoch 85
Validation binary_cross_entropy = 0.639152
Epoch 86
Validation binary_cross_entropy = 0.620365
Epoch 87
Validation binary_cross_entropy = 0.614284
Epoch 88
Validation binary_cross_entropy = 0.624715
Epoch 89
Loss = 7.5236e-01, PNorm = 27.5593, GNorm = 11.4457, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.627227
Epoch 90
Validation binary_cross_entropy = 0.655245
Epoch 91
Validation binary_cross_entropy = 0.658511
Epoch 92
Validation binary_cross_entropy = 0.645916
Epoch 93
Validation binary_cross_entropy = 0.638797
Epoch 94
Loss = 6.1088e-01, PNorm = 27.5652, GNorm = 8.5192, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.619644
Epoch 95
Validation binary_cross_entropy = 0.597896
Epoch 96
Validation binary_cross_entropy = 0.588918
Epoch 97
Validation binary_cross_entropy = 0.554147
Epoch 98
Validation binary_cross_entropy = 0.535276
Epoch 99
Loss = 9.3502e-01, PNorm = 27.5706, GNorm = 10.6167, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.526297
Epoch 100
Validation binary_cross_entropy = 0.547544
Epoch 101
Validation binary_cross_entropy = 0.584505
Epoch 102
Validation binary_cross_entropy = 0.615951
Epoch 103
Validation binary_cross_entropy = 0.638559
Epoch 104
Loss = 6.5498e-01, PNorm = 27.5756, GNorm = 8.9356, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.640185
Epoch 105
Validation binary_cross_entropy = 0.637676
Epoch 106
Validation binary_cross_entropy = 0.622599
Epoch 107
Validation binary_cross_entropy = 0.585618
Epoch 108
Validation binary_cross_entropy = 0.568240
Epoch 109
Loss = 5.8946e-01, PNorm = 27.5808, GNorm = 9.9496, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.557190
Epoch 110
Validation binary_cross_entropy = 0.549754
Epoch 111
Validation binary_cross_entropy = 0.551996
Epoch 112
Validation binary_cross_entropy = 0.565866
Epoch 113
Validation binary_cross_entropy = 0.586655
Epoch 114
Loss = 5.9828e-01, PNorm = 27.5856, GNorm = 6.3971, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.586815
Epoch 115
Validation binary_cross_entropy = 0.582667
Epoch 116
Validation binary_cross_entropy = 0.589201
Epoch 117
Validation binary_cross_entropy = 0.590011
Epoch 118
Validation binary_cross_entropy = 0.593610
Epoch 119
Loss = 7.0955e-01, PNorm = 27.5902, GNorm = 8.8222, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.595346
Epoch 120
Validation binary_cross_entropy = 0.590933
Epoch 121
Validation binary_cross_entropy = 0.589670
Epoch 122
Validation binary_cross_entropy = 0.593768
Epoch 123
Validation binary_cross_entropy = 0.598168
Epoch 124
Loss = 6.2997e-01, PNorm = 27.5946, GNorm = 6.7238, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.598685
Epoch 125
Validation binary_cross_entropy = 0.579710
Epoch 126
Validation binary_cross_entropy = 0.554476
Epoch 127
Validation binary_cross_entropy = 0.541299
Epoch 128
Validation binary_cross_entropy = 0.555519
Epoch 129
Loss = 4.6981e-01, PNorm = 27.5995, GNorm = 5.0710, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.565728
Epoch 130
Validation binary_cross_entropy = 0.587159
Epoch 131
Validation binary_cross_entropy = 0.590460
Epoch 132
Validation binary_cross_entropy = 0.570577
Epoch 133
Validation binary_cross_entropy = 0.559448
Epoch 134
Loss = 4.1519e-01, PNorm = 27.6046, GNorm = 6.8379, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.570977
Epoch 135
Validation binary_cross_entropy = 0.585303
Epoch 136
Validation binary_cross_entropy = 0.588781
Epoch 137
Validation binary_cross_entropy = 0.590498
Epoch 138
Validation binary_cross_entropy = 0.591150
Epoch 139
Loss = 7.1889e-01, PNorm = 27.6093, GNorm = 7.1566, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.581035
Epoch 140
Validation binary_cross_entropy = 0.569409
Epoch 141
Validation binary_cross_entropy = 0.572222
Epoch 142
Validation binary_cross_entropy = 0.574758
Epoch 143
Validation binary_cross_entropy = 0.580481
Epoch 144
Loss = 6.0933e-01, PNorm = 27.6129, GNorm = 6.2669, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.585926
Epoch 145
Validation binary_cross_entropy = 0.601368
Epoch 146
Validation binary_cross_entropy = 0.623277
Epoch 147
Validation binary_cross_entropy = 0.642358
Epoch 148
Validation binary_cross_entropy = 0.651980
Epoch 149
Loss = 5.7669e-01, PNorm = 27.6162, GNorm = 8.8589, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.666747
Epoch 150
Validation binary_cross_entropy = 0.652850
Epoch 151
Validation binary_cross_entropy = 0.642269
Epoch 152
Validation binary_cross_entropy = 0.624268
Epoch 153
Validation binary_cross_entropy = 0.608038
Epoch 154
Loss = 6.2361e-01, PNorm = 27.6208, GNorm = 9.4991, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.589587
Epoch 155
Validation binary_cross_entropy = 0.577082
Epoch 156
Validation binary_cross_entropy = 0.573406
Epoch 157
Validation binary_cross_entropy = 0.572775
Epoch 158
Validation binary_cross_entropy = 0.577059
Epoch 159
Loss = 4.6230e-01, PNorm = 27.6263, GNorm = 5.5056, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.581812
Epoch 160
Validation binary_cross_entropy = 0.583270
Epoch 161
Validation binary_cross_entropy = 0.585504
Epoch 162
Validation binary_cross_entropy = 0.587742
Epoch 163
Validation binary_cross_entropy = 0.587703
Epoch 164
Loss = 4.9786e-01, PNorm = 27.6321, GNorm = 6.0005, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.596889
Epoch 165
Validation binary_cross_entropy = 0.601471
Epoch 166
Validation binary_cross_entropy = 0.602987
Epoch 167
Validation binary_cross_entropy = 0.605279
Epoch 168
Validation binary_cross_entropy = 0.609430
Epoch 169
Loss = 6.8641e-01, PNorm = 27.6362, GNorm = 8.1382, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.612731
Epoch 170
Validation binary_cross_entropy = 0.611387
Epoch 171
Validation binary_cross_entropy = 0.617182
Epoch 172
Validation binary_cross_entropy = 0.622205
Epoch 173
Validation binary_cross_entropy = 0.614097
Epoch 174
Loss = 5.7883e-01, PNorm = 27.6392, GNorm = 6.7010, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.605190
Epoch 175
Validation binary_cross_entropy = 0.595819
Epoch 176
Validation binary_cross_entropy = 0.587197
Epoch 177
Validation binary_cross_entropy = 0.580358
Epoch 178
Validation binary_cross_entropy = 0.583559
Epoch 179
Loss = 4.9421e-01, PNorm = 27.6437, GNorm = 5.8317, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.601422
Epoch 180
Validation binary_cross_entropy = 0.620206
Epoch 181
Validation binary_cross_entropy = 0.632613
Epoch 182
Validation binary_cross_entropy = 0.633480
Epoch 183
Validation binary_cross_entropy = 0.630208
Epoch 184
Loss = 4.8808e-01, PNorm = 27.6492, GNorm = 6.2503, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.627229
Epoch 185
Validation binary_cross_entropy = 0.608754
Epoch 186
Validation binary_cross_entropy = 0.593336
Epoch 187
Validation binary_cross_entropy = 0.577421
Epoch 188
Validation binary_cross_entropy = 0.568579
Epoch 189
Loss = 5.9127e-01, PNorm = 27.6539, GNorm = 5.8923, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.568278
Epoch 190
Validation binary_cross_entropy = 0.573994
Epoch 191
Validation binary_cross_entropy = 0.577991
Epoch 192
Validation binary_cross_entropy = 0.584931
Epoch 193
Validation binary_cross_entropy = 0.587779
Epoch 194
Loss = 6.1204e-01, PNorm = 27.6578, GNorm = 5.5428, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.582569
Epoch 195
Validation binary_cross_entropy = 0.581173
Epoch 196
Validation binary_cross_entropy = 0.583834
Epoch 197
Validation binary_cross_entropy = 0.591682
Epoch 198
Validation binary_cross_entropy = 0.601463
Epoch 199
Loss = 6.3793e-01, PNorm = 27.6622, GNorm = 7.0087, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.597723
Epoch 200
Validation binary_cross_entropy = 0.597250
Epoch 201
Validation binary_cross_entropy = 0.606435
Epoch 202
Validation binary_cross_entropy = 0.606953
Epoch 203
Validation binary_cross_entropy = 0.610138
Epoch 204
Loss = 5.7324e-01, PNorm = 27.6666, GNorm = 5.8519, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.613545
Epoch 205
Validation binary_cross_entropy = 0.610487
Epoch 206
Validation binary_cross_entropy = 0.612244
Epoch 207
Validation binary_cross_entropy = 0.611321
Epoch 208
Validation binary_cross_entropy = 0.608320
Epoch 209
Loss = 4.8570e-01, PNorm = 27.6706, GNorm = 6.5269, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.602183
Epoch 210
Validation binary_cross_entropy = 0.597667
Epoch 211
Validation binary_cross_entropy = 0.604423
Epoch 212
Validation binary_cross_entropy = 0.606368
Epoch 213
Validation binary_cross_entropy = 0.610046
Epoch 214
Loss = 4.8125e-01, PNorm = 27.6758, GNorm = 5.3445, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.615546
Epoch 215
Validation binary_cross_entropy = 0.623826
Epoch 216
Validation binary_cross_entropy = 0.628038
Epoch 217
Validation binary_cross_entropy = 0.635759
Epoch 218
Validation binary_cross_entropy = 0.640521
Epoch 219
Loss = 5.8342e-01, PNorm = 27.6811, GNorm = 6.7876, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.637890
Epoch 220
Validation binary_cross_entropy = 0.639686
Epoch 221
Validation binary_cross_entropy = 0.648370
Epoch 222
Validation binary_cross_entropy = 0.654563
Epoch 223
Validation binary_cross_entropy = 0.653504
Epoch 224
Loss = 5.2670e-01, PNorm = 27.6848, GNorm = 4.9054, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.641450
Epoch 225
Validation binary_cross_entropy = 0.633671
Epoch 226
Validation binary_cross_entropy = 0.635845
Epoch 227
Validation binary_cross_entropy = 0.636466
Epoch 228
Validation binary_cross_entropy = 0.637225
Epoch 229
Loss = 3.9487e-01, PNorm = 27.6879, GNorm = 8.9383, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.637797
Epoch 230
Validation binary_cross_entropy = 0.628953
Epoch 231
Validation binary_cross_entropy = 0.624182
Epoch 232
Validation binary_cross_entropy = 0.619228
Epoch 233
Validation binary_cross_entropy = 0.612302
Epoch 234
Loss = 4.9111e-01, PNorm = 27.6922, GNorm = 8.4597, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.610925
Epoch 235
Validation binary_cross_entropy = 0.621905
Epoch 236
Validation binary_cross_entropy = 0.632671
Epoch 237
Validation binary_cross_entropy = 0.639405
Epoch 238
Validation binary_cross_entropy = 0.643374
Epoch 239
Loss = 3.5791e-01, PNorm = 27.6971, GNorm = 3.3995, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.638720
Epoch 240
Validation binary_cross_entropy = 0.639692
Epoch 241
Validation binary_cross_entropy = 0.633135
Epoch 242
Validation binary_cross_entropy = 0.631908
Epoch 243
Validation binary_cross_entropy = 0.627532
Epoch 244
Loss = 5.0320e-01, PNorm = 27.7014, GNorm = 4.6140, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.618486
Epoch 245
Validation binary_cross_entropy = 0.610544
Epoch 246
Validation binary_cross_entropy = 0.608346
Epoch 247
Validation binary_cross_entropy = 0.613426
Epoch 248
Validation binary_cross_entropy = 0.616991
Epoch 249
Loss = 5.3146e-01, PNorm = 27.7061, GNorm = 3.3849, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.623171
Epoch 250
Validation binary_cross_entropy = 0.636425
Epoch 251
Validation binary_cross_entropy = 0.649045
Epoch 252
Validation binary_cross_entropy = 0.660432
Epoch 253
Validation binary_cross_entropy = 0.644965
Epoch 254
Loss = 5.0831e-01, PNorm = 27.7099, GNorm = 3.7973, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.636554
Epoch 255
Validation binary_cross_entropy = 0.627433
Epoch 256
Validation binary_cross_entropy = 0.617002
Epoch 257
Validation binary_cross_entropy = 0.611021
Epoch 258
Validation binary_cross_entropy = 0.599839
Epoch 259
Loss = 5.0739e-01, PNorm = 27.7139, GNorm = 4.8610, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.591753
Epoch 260
Validation binary_cross_entropy = 0.582689
Epoch 261
Validation binary_cross_entropy = 0.575447
Epoch 262
Validation binary_cross_entropy = 0.570622
Epoch 263
Validation binary_cross_entropy = 0.567671
Epoch 264
Loss = 4.6134e-01, PNorm = 27.7178, GNorm = 3.9295, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.567517
Epoch 265
Validation binary_cross_entropy = 0.567429
Epoch 266
Validation binary_cross_entropy = 0.566361
Epoch 267
Validation binary_cross_entropy = 0.568541
Epoch 268
Validation binary_cross_entropy = 0.575506
Epoch 269
Loss = 4.9487e-01, PNorm = 27.7215, GNorm = 8.3603, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.576235
Epoch 270
Validation binary_cross_entropy = 0.577343
Epoch 271
Validation binary_cross_entropy = 0.568393
Epoch 272
Validation binary_cross_entropy = 0.560866
Epoch 273
Validation binary_cross_entropy = 0.562989
Epoch 274
Loss = 5.0002e-01, PNorm = 27.7258, GNorm = 5.6663, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.574105
Epoch 275
Validation binary_cross_entropy = 0.586857
Epoch 276
Validation binary_cross_entropy = 0.603804
Epoch 277
Validation binary_cross_entropy = 0.620319
Epoch 278
Validation binary_cross_entropy = 0.632966
Epoch 279
Loss = 5.3069e-01, PNorm = 27.7297, GNorm = 5.3468, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.639374
Epoch 280
Validation binary_cross_entropy = 0.629846
Epoch 281
Validation binary_cross_entropy = 0.619520
Epoch 282
Validation binary_cross_entropy = 0.609560
Epoch 283
Validation binary_cross_entropy = 0.610505
Epoch 284
Loss = 5.3673e-01, PNorm = 27.7332, GNorm = 4.7553, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.606866
Epoch 285
Validation binary_cross_entropy = 0.601221
Epoch 286
Validation binary_cross_entropy = 0.592401
Epoch 287
Validation binary_cross_entropy = 0.589412
Epoch 288
Validation binary_cross_entropy = 0.597790
Epoch 289
Loss = 4.4153e-01, PNorm = 27.7373, GNorm = 4.9938, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.606428
Epoch 290
Validation binary_cross_entropy = 0.618565
Epoch 291
Validation binary_cross_entropy = 0.625809
Epoch 292
Validation binary_cross_entropy = 0.621448
Epoch 293
Validation binary_cross_entropy = 0.612193
Epoch 294
Loss = 6.4895e-01, PNorm = 27.7418, GNorm = 4.1551, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.594726
Epoch 295
Validation binary_cross_entropy = 0.580831
Epoch 296
Validation binary_cross_entropy = 0.569458
Epoch 297
Validation binary_cross_entropy = 0.563465
Epoch 298
Validation binary_cross_entropy = 0.559624
Epoch 299
Loss = 3.5191e-01, PNorm = 27.7459, GNorm = 4.7921, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.563960
Model 0 best validation binary_cross_entropy = 0.338497 on epoch 4
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.272460
Ensemble test binary_cross_entropy = 0.272460
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 281,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.103007
Epoch 1
Validation binary_cross_entropy = 2.365136
Epoch 2
Validation binary_cross_entropy = 1.164554
Epoch 3
Validation binary_cross_entropy = 0.406706
Epoch 4
Loss = 3.7874e+00, PNorm = 27.3557, GNorm = 52.9316, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.488234
Epoch 5
Validation binary_cross_entropy = 0.517431
Epoch 6
Validation binary_cross_entropy = 0.886524
Epoch 7
Validation binary_cross_entropy = 0.675070
Epoch 8
Validation binary_cross_entropy = 0.572416
Epoch 9
Loss = 2.2623e+00, PNorm = 27.3736, GNorm = 32.2541, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.508574
Epoch 10
Validation binary_cross_entropy = 0.513640
Epoch 11
Validation binary_cross_entropy = 0.556134
Epoch 12
Validation binary_cross_entropy = 0.813003
Epoch 13
Validation binary_cross_entropy = 0.831969
Epoch 14
Loss = 2.3284e+00, PNorm = 27.4039, GNorm = 22.3468, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 1.333915
Epoch 15
Validation binary_cross_entropy = 0.963718
Epoch 16
Validation binary_cross_entropy = 0.647988
Epoch 17
Validation binary_cross_entropy = 0.473849
Epoch 18
Validation binary_cross_entropy = 0.622426
Epoch 19
Loss = 1.9325e+00, PNorm = 27.4304, GNorm = 24.5954, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 1.464950
Epoch 20
Validation binary_cross_entropy = 2.582591
Epoch 21
Validation binary_cross_entropy = 2.673889
Epoch 22
Validation binary_cross_entropy = 1.483595
Epoch 23
Validation binary_cross_entropy = 0.406101
Epoch 24
Loss = 2.1005e+00, PNorm = 27.4525, GNorm = 31.3318, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.342718
Epoch 25
Validation binary_cross_entropy = 0.406734
Epoch 26
Validation binary_cross_entropy = 1.387379
Epoch 27
Validation binary_cross_entropy = 1.961499
Epoch 28
Validation binary_cross_entropy = 1.565009
Epoch 29
Loss = 1.9242e+00, PNorm = 27.4708, GNorm = 21.2946, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.543425
Epoch 30
Validation binary_cross_entropy = 0.343740
Epoch 31
Validation binary_cross_entropy = 0.341142
Epoch 32
Validation binary_cross_entropy = 0.372986
Epoch 33
Validation binary_cross_entropy = 0.572890
Epoch 34
Loss = 1.2984e+00, PNorm = 27.4866, GNorm = 20.6706, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.962960
Epoch 35
Validation binary_cross_entropy = 1.090202
Epoch 36
Validation binary_cross_entropy = 0.625915
Epoch 37
Validation binary_cross_entropy = 0.365615
Epoch 38
Validation binary_cross_entropy = 0.361803
Epoch 39
Loss = 1.1285e+00, PNorm = 27.4994, GNorm = 21.4167, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.453306
Epoch 40
Validation binary_cross_entropy = 0.719686
Epoch 41
Validation binary_cross_entropy = 1.034562
Epoch 42
Validation binary_cross_entropy = 1.067771
Epoch 43
Validation binary_cross_entropy = 0.949625
Epoch 44
Loss = 1.2369e+00, PNorm = 27.5094, GNorm = 13.7144, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.766442
Epoch 45
Validation binary_cross_entropy = 0.725496
Epoch 46
Validation binary_cross_entropy = 0.760835
Epoch 47
Validation binary_cross_entropy = 0.885138
Epoch 48
Validation binary_cross_entropy = 0.923761
Epoch 49
Loss = 1.5044e+00, PNorm = 27.5183, GNorm = 15.7942, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.970214
Epoch 50
Validation binary_cross_entropy = 1.090938
Epoch 51
Validation binary_cross_entropy = 1.108330
Epoch 52
Validation binary_cross_entropy = 1.067363
Epoch 53
Validation binary_cross_entropy = 0.936361
Epoch 54
Loss = 1.0674e+00, PNorm = 27.5262, GNorm = 16.7122, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.803314
Epoch 55
Validation binary_cross_entropy = 0.649528
Epoch 56
Validation binary_cross_entropy = 0.563031
Epoch 57
Validation binary_cross_entropy = 0.566327
Epoch 58
Validation binary_cross_entropy = 0.568755
Epoch 59
Loss = 1.1567e+00, PNorm = 27.5331, GNorm = 12.3348, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.598201
Epoch 60
Validation binary_cross_entropy = 0.635574
Epoch 61
Validation binary_cross_entropy = 0.684503
Epoch 62
Validation binary_cross_entropy = 0.668848
Epoch 63
Validation binary_cross_entropy = 0.606728
Epoch 64
Loss = 8.6857e-01, PNorm = 27.5396, GNorm = 13.6159, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.610446
Epoch 65
Validation binary_cross_entropy = 0.648671
Epoch 66
Validation binary_cross_entropy = 0.736321
Epoch 67
Validation binary_cross_entropy = 0.826000
Epoch 68
Validation binary_cross_entropy = 0.949039
Epoch 69
Loss = 6.7232e-01, PNorm = 27.5461, GNorm = 11.7031, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 1.026746
Epoch 70
Validation binary_cross_entropy = 0.931911
Epoch 71
Validation binary_cross_entropy = 0.844792
Epoch 72
Validation binary_cross_entropy = 0.801972
Epoch 73
Validation binary_cross_entropy = 0.786519
Epoch 74
Loss = 8.3606e-01, PNorm = 27.5511, GNorm = 11.1469, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.795716
Epoch 75
Validation binary_cross_entropy = 0.813333
Epoch 76
Validation binary_cross_entropy = 0.903195
Epoch 77
Validation binary_cross_entropy = 0.891316
Epoch 78
Validation binary_cross_entropy = 0.872877
Epoch 79
Loss = 9.6307e-01, PNorm = 27.5557, GNorm = 13.3804, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.783978
Epoch 80
Validation binary_cross_entropy = 0.705719
Epoch 81
Validation binary_cross_entropy = 0.615764
Epoch 82
Validation binary_cross_entropy = 0.530441
Epoch 83
Validation binary_cross_entropy = 0.494853
Epoch 84
Loss = 8.6107e-01, PNorm = 27.5608, GNorm = 11.1983, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.502661
Epoch 85
Validation binary_cross_entropy = 0.564312
Epoch 86
Validation binary_cross_entropy = 0.617113
Epoch 87
Validation binary_cross_entropy = 0.641262
Epoch 88
Validation binary_cross_entropy = 0.625447
Epoch 89
Loss = 8.3164e-01, PNorm = 27.5669, GNorm = 12.5167, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.571460
Epoch 90
Validation binary_cross_entropy = 0.575177
Epoch 91
Validation binary_cross_entropy = 0.595601
Epoch 92
Validation binary_cross_entropy = 0.581722
Epoch 93
Validation binary_cross_entropy = 0.575340
Epoch 94
Loss = 7.6797e-01, PNorm = 27.5714, GNorm = 12.1313, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.562535
Epoch 95
Validation binary_cross_entropy = 0.601060
Epoch 96
Validation binary_cross_entropy = 0.646140
Epoch 97
Validation binary_cross_entropy = 0.691599
Epoch 98
Validation binary_cross_entropy = 0.719214
Epoch 99
Loss = 8.5442e-01, PNorm = 27.5761, GNorm = 10.9378, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.723967
Epoch 100
Validation binary_cross_entropy = 0.683879
Epoch 101
Validation binary_cross_entropy = 0.648995
Epoch 102
Validation binary_cross_entropy = 0.620989
Epoch 103
Validation binary_cross_entropy = 0.603756
Epoch 104
Loss = 5.8500e-01, PNorm = 27.5806, GNorm = 6.6762, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.589707
Epoch 105
Validation binary_cross_entropy = 0.587143
Epoch 106
Validation binary_cross_entropy = 0.604014
Epoch 107
Validation binary_cross_entropy = 0.601477
Epoch 108
Validation binary_cross_entropy = 0.587266
Epoch 109
Loss = 7.4395e-01, PNorm = 27.5831, GNorm = 10.2664, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.566975
Epoch 110
Validation binary_cross_entropy = 0.547184
Epoch 111
Validation binary_cross_entropy = 0.540144
Epoch 112
Validation binary_cross_entropy = 0.544551
Epoch 113
Validation binary_cross_entropy = 0.548115
Epoch 114
Loss = 7.1799e-01, PNorm = 27.5857, GNorm = 7.5319, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.554382
Epoch 115
Validation binary_cross_entropy = 0.585167
Epoch 116
Validation binary_cross_entropy = 0.619897
Epoch 117
Validation binary_cross_entropy = 0.643022
Epoch 118
Validation binary_cross_entropy = 0.662596
Epoch 119
Loss = 6.4996e-01, PNorm = 27.5897, GNorm = 5.2725, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.656220
Epoch 120
Validation binary_cross_entropy = 0.653507
Epoch 121
Validation binary_cross_entropy = 0.614270
Epoch 122
Validation binary_cross_entropy = 0.552855
Epoch 123
Validation binary_cross_entropy = 0.515119
Epoch 124
Loss = 8.0706e-01, PNorm = 27.5944, GNorm = 9.1196, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.490558
Epoch 125
Validation binary_cross_entropy = 0.483611
Epoch 126
Validation binary_cross_entropy = 0.488255
Epoch 127
Validation binary_cross_entropy = 0.505878
Epoch 128
Validation binary_cross_entropy = 0.542888
Epoch 129
Loss = 7.4493e-01, PNorm = 27.5986, GNorm = 9.0828, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.585329
Epoch 130
Validation binary_cross_entropy = 0.615693
Epoch 131
Validation binary_cross_entropy = 0.639633
Epoch 132
Validation binary_cross_entropy = 0.654155
Epoch 133
Validation binary_cross_entropy = 0.656693
Epoch 134
Loss = 6.4559e-01, PNorm = 27.6034, GNorm = 7.2161, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.649658
Epoch 135
Validation binary_cross_entropy = 0.628875
Epoch 136
Validation binary_cross_entropy = 0.602737
Epoch 137
Validation binary_cross_entropy = 0.576647
Epoch 138
Validation binary_cross_entropy = 0.565520
Epoch 139
Loss = 6.6813e-01, PNorm = 27.6083, GNorm = 8.1027, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.559059
Epoch 140
Validation binary_cross_entropy = 0.557709
Epoch 141
Validation binary_cross_entropy = 0.557287
Epoch 142
Validation binary_cross_entropy = 0.561826
Epoch 143
Validation binary_cross_entropy = 0.567860
Epoch 144
Loss = 5.7993e-01, PNorm = 27.6126, GNorm = 5.6398, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.565807
Epoch 145
Validation binary_cross_entropy = 0.564290
Epoch 146
Validation binary_cross_entropy = 0.568042
Epoch 147
Validation binary_cross_entropy = 0.572275
Epoch 148
Validation binary_cross_entropy = 0.579529
Epoch 149
Loss = 6.6899e-01, PNorm = 27.6172, GNorm = 8.3975, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.577844
Epoch 150
Validation binary_cross_entropy = 0.563481
Epoch 151
Validation binary_cross_entropy = 0.554850
Epoch 152
Validation binary_cross_entropy = 0.541608
Epoch 153
Validation binary_cross_entropy = 0.531492
Epoch 154
Loss = 6.3883e-01, PNorm = 27.6219, GNorm = 7.9042, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.520158
Epoch 155
Validation binary_cross_entropy = 0.525849
Epoch 156
Validation binary_cross_entropy = 0.542281
Epoch 157
Validation binary_cross_entropy = 0.550495
Epoch 158
Validation binary_cross_entropy = 0.562298
Epoch 159
Loss = 5.0495e-01, PNorm = 27.6264, GNorm = 5.9375, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.571623
Epoch 160
Validation binary_cross_entropy = 0.577233
Epoch 161
Validation binary_cross_entropy = 0.591312
Epoch 162
Validation binary_cross_entropy = 0.602023
Epoch 163
Validation binary_cross_entropy = 0.604496
Epoch 164
Loss = 5.5400e-01, PNorm = 27.6305, GNorm = 7.6929, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.601461
Epoch 165
Validation binary_cross_entropy = 0.597084
Epoch 166
Validation binary_cross_entropy = 0.586554
Epoch 167
Validation binary_cross_entropy = 0.584225
Epoch 168
Validation binary_cross_entropy = 0.581427
Epoch 169
Loss = 5.3853e-01, PNorm = 27.6335, GNorm = 6.9883, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.576959
Epoch 170
Validation binary_cross_entropy = 0.572731
Epoch 171
Validation binary_cross_entropy = 0.547596
Epoch 172
Validation binary_cross_entropy = 0.529386
Epoch 173
Validation binary_cross_entropy = 0.513044
Epoch 174
Loss = 6.2532e-01, PNorm = 27.6374, GNorm = 5.2535, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.499476
Epoch 175
Validation binary_cross_entropy = 0.500605
Epoch 176
Validation binary_cross_entropy = 0.511195
Epoch 177
Validation binary_cross_entropy = 0.519297
Epoch 178
Validation binary_cross_entropy = 0.516417
Epoch 179
Loss = 5.6366e-01, PNorm = 27.6412, GNorm = 6.0062, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.507236
Epoch 180
Validation binary_cross_entropy = 0.503958
Epoch 181
Validation binary_cross_entropy = 0.499854
Epoch 182
Validation binary_cross_entropy = 0.506079
Epoch 183
Validation binary_cross_entropy = 0.516655
Epoch 184
Loss = 4.5991e-01, PNorm = 27.6451, GNorm = 5.6978, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.530045
Epoch 185
Validation binary_cross_entropy = 0.545668
Epoch 186
Validation binary_cross_entropy = 0.561810
Epoch 187
Validation binary_cross_entropy = 0.566736
Epoch 188
Validation binary_cross_entropy = 0.564918
Epoch 189
Loss = 5.9172e-01, PNorm = 27.6489, GNorm = 5.4624, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.564478
Epoch 190
Validation binary_cross_entropy = 0.554955
Epoch 191
Validation binary_cross_entropy = 0.544050
Epoch 192
Validation binary_cross_entropy = 0.542541
Epoch 193
Validation binary_cross_entropy = 0.541194
Epoch 194
Loss = 5.9179e-01, PNorm = 27.6523, GNorm = 8.0712, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.532795
Epoch 195
Validation binary_cross_entropy = 0.525511
Epoch 196
Validation binary_cross_entropy = 0.521302
Epoch 197
Validation binary_cross_entropy = 0.513458
Epoch 198
Validation binary_cross_entropy = 0.512316
Epoch 199
Loss = 6.5217e-01, PNorm = 27.6556, GNorm = 7.0636, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.524185
Epoch 200
Validation binary_cross_entropy = 0.534893
Epoch 201
Validation binary_cross_entropy = 0.552796
Epoch 202
Validation binary_cross_entropy = 0.574782
Epoch 203
Validation binary_cross_entropy = 0.593896
Epoch 204
Loss = 4.8558e-01, PNorm = 27.6595, GNorm = 4.8092, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.610736
Epoch 205
Validation binary_cross_entropy = 0.618006
Epoch 206
Validation binary_cross_entropy = 0.607458
Epoch 207
Validation binary_cross_entropy = 0.597584
Epoch 208
Validation binary_cross_entropy = 0.586937
Epoch 209
Loss = 4.4323e-01, PNorm = 27.6631, GNorm = 4.9980, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.582015
Epoch 210
Validation binary_cross_entropy = 0.584192
Epoch 211
Validation binary_cross_entropy = 0.589073
Epoch 212
Validation binary_cross_entropy = 0.591166
Epoch 213
Validation binary_cross_entropy = 0.585946
Epoch 214
Loss = 5.5303e-01, PNorm = 27.6670, GNorm = 5.0891, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.581320
Epoch 215
Validation binary_cross_entropy = 0.570109
Epoch 216
Validation binary_cross_entropy = 0.563962
Epoch 217
Validation binary_cross_entropy = 0.559493
Epoch 218
Validation binary_cross_entropy = 0.561725
Epoch 219
Loss = 5.5584e-01, PNorm = 27.6707, GNorm = 4.7289, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.570958
Epoch 220
Validation binary_cross_entropy = 0.576375
Epoch 221
Validation binary_cross_entropy = 0.585348
Epoch 222
Validation binary_cross_entropy = 0.593623
Epoch 223
Validation binary_cross_entropy = 0.596238
Epoch 224
Loss = 4.2305e-01, PNorm = 27.6746, GNorm = 5.4992, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.594053
Epoch 225
Validation binary_cross_entropy = 0.587315
Epoch 226
Validation binary_cross_entropy = 0.585406
Epoch 227
Validation binary_cross_entropy = 0.579988
Epoch 228
Validation binary_cross_entropy = 0.574502
Epoch 229
Loss = 6.2965e-01, PNorm = 27.6783, GNorm = 5.9454, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.561179
Epoch 230
Validation binary_cross_entropy = 0.553865
Epoch 231
Validation binary_cross_entropy = 0.549620
Epoch 232
Validation binary_cross_entropy = 0.547950
Epoch 233
Validation binary_cross_entropy = 0.548849
Epoch 234
Loss = 5.3014e-01, PNorm = 27.6826, GNorm = 5.8704, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.551736
Epoch 235
Validation binary_cross_entropy = 0.565232
Epoch 236
Validation binary_cross_entropy = 0.579293
Epoch 237
Validation binary_cross_entropy = 0.601741
Epoch 238
Validation binary_cross_entropy = 0.625033
Epoch 239
Loss = 4.4545e-01, PNorm = 27.6860, GNorm = 6.3436, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.634069
Epoch 240
Validation binary_cross_entropy = 0.632916
Epoch 241
Validation binary_cross_entropy = 0.621799
Epoch 242
Validation binary_cross_entropy = 0.606176
Epoch 243
Validation binary_cross_entropy = 0.591462
Epoch 244
Loss = 6.1684e-01, PNorm = 27.6888, GNorm = 7.3100, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.578876
Epoch 245
Validation binary_cross_entropy = 0.573258
Epoch 246
Validation binary_cross_entropy = 0.563290
Epoch 247
Validation binary_cross_entropy = 0.563460
Epoch 248
Validation binary_cross_entropy = 0.563200
Epoch 249
Loss = 4.4926e-01, PNorm = 27.6926, GNorm = 5.0805, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.560959
Epoch 250
Validation binary_cross_entropy = 0.561235
Epoch 251
Validation binary_cross_entropy = 0.563273
Epoch 252
Validation binary_cross_entropy = 0.566515
Epoch 253
Validation binary_cross_entropy = 0.563992
Epoch 254
Loss = 5.5092e-01, PNorm = 27.6962, GNorm = 5.3851, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.558232
Epoch 255
Validation binary_cross_entropy = 0.548383
Epoch 256
Validation binary_cross_entropy = 0.539199
Epoch 257
Validation binary_cross_entropy = 0.536215
Epoch 258
Validation binary_cross_entropy = 0.539509
Epoch 259
Loss = 5.2503e-01, PNorm = 27.6992, GNorm = 5.1307, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.545299
Epoch 260
Validation binary_cross_entropy = 0.552736
Epoch 261
Validation binary_cross_entropy = 0.563035
Epoch 262
Validation binary_cross_entropy = 0.568585
Epoch 263
Validation binary_cross_entropy = 0.570519
Epoch 264
Loss = 5.1465e-01, PNorm = 27.7026, GNorm = 4.8717, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.572631
Epoch 265
Validation binary_cross_entropy = 0.578354
Epoch 266
Validation binary_cross_entropy = 0.581491
Epoch 267
Validation binary_cross_entropy = 0.580807
Epoch 268
Validation binary_cross_entropy = 0.577752
Epoch 269
Loss = 5.2256e-01, PNorm = 27.7052, GNorm = 5.0235, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.577636
Epoch 270
Validation binary_cross_entropy = 0.579537
Epoch 271
Validation binary_cross_entropy = 0.579185
Epoch 272
Validation binary_cross_entropy = 0.586735
Epoch 273
Validation binary_cross_entropy = 0.590170
Epoch 274
Loss = 3.4196e-01, PNorm = 27.7083, GNorm = 4.6580, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.588486
Epoch 275
Validation binary_cross_entropy = 0.584090
Epoch 276
Validation binary_cross_entropy = 0.577518
Epoch 277
Validation binary_cross_entropy = 0.569579
Epoch 278
Validation binary_cross_entropy = 0.565653
Epoch 279
Loss = 4.9779e-01, PNorm = 27.7115, GNorm = 5.1043, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.565561
Epoch 280
Validation binary_cross_entropy = 0.562214
Epoch 281
Validation binary_cross_entropy = 0.559685
Epoch 282
Validation binary_cross_entropy = 0.564919
Epoch 283
Validation binary_cross_entropy = 0.573507
Epoch 284
Loss = 4.9750e-01, PNorm = 27.7148, GNorm = 4.2588, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.579542
Epoch 285
Validation binary_cross_entropy = 0.587255
Epoch 286
Validation binary_cross_entropy = 0.591875
Epoch 287
Validation binary_cross_entropy = 0.597775
Epoch 288
Validation binary_cross_entropy = 0.596870
Epoch 289
Loss = 4.2217e-01, PNorm = 27.7182, GNorm = 3.1441, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.596164
Epoch 290
Validation binary_cross_entropy = 0.590673
Epoch 291
Validation binary_cross_entropy = 0.589494
Epoch 292
Validation binary_cross_entropy = 0.593161
Epoch 293
Validation binary_cross_entropy = 0.596407
Epoch 294
Loss = 4.3319e-01, PNorm = 27.7216, GNorm = 4.0518, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.596240
Epoch 295
Validation binary_cross_entropy = 0.597428
Epoch 296
Validation binary_cross_entropy = 0.603446
Epoch 297
Validation binary_cross_entropy = 0.611998
Epoch 298
Validation binary_cross_entropy = 0.610987
Epoch 299
Loss = 5.8413e-01, PNorm = 27.7249, GNorm = 6.1232, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.610182
Model 0 best validation binary_cross_entropy = 0.341142 on epoch 31
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.226135
Ensemble test binary_cross_entropy = 0.226135
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 281,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.997291
Epoch 1
Validation binary_cross_entropy = 1.749157
Epoch 2
Validation binary_cross_entropy = 1.298840
Epoch 3
Validation binary_cross_entropy = 0.413188
Epoch 4
Loss = 4.9042e+00, PNorm = 27.3563, GNorm = 53.5771, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.397054
Epoch 5
Validation binary_cross_entropy = 0.602860
Epoch 6
Validation binary_cross_entropy = 1.129475
Epoch 7
Validation binary_cross_entropy = 0.497320
Epoch 8
Validation binary_cross_entropy = 0.439569
Epoch 9
Loss = 2.9523e+00, PNorm = 27.3725, GNorm = 40.8855, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.683155
Epoch 10
Validation binary_cross_entropy = 1.814584
Epoch 11
Validation binary_cross_entropy = 1.209919
Epoch 12
Validation binary_cross_entropy = 0.648807
Epoch 13
Validation binary_cross_entropy = 0.721992
Epoch 14
Loss = 1.8447e+00, PNorm = 27.4002, GNorm = 36.7303, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 1.142151
Epoch 15
Validation binary_cross_entropy = 1.559138
Epoch 16
Validation binary_cross_entropy = 2.022502
Epoch 17
Validation binary_cross_entropy = 1.955480
Epoch 18
Validation binary_cross_entropy = 1.084905
Epoch 19
Loss = 2.1658e+00, PNorm = 27.4232, GNorm = 33.7921, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.659773
Epoch 20
Validation binary_cross_entropy = 0.999759
Epoch 21
Validation binary_cross_entropy = 1.916128
Epoch 22
Validation binary_cross_entropy = 1.555006
Epoch 23
Validation binary_cross_entropy = 0.679996
Epoch 24
Loss = 1.7478e+00, PNorm = 27.4419, GNorm = 21.1165, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.395691
Epoch 25
Validation binary_cross_entropy = 0.384625
Epoch 26
Validation binary_cross_entropy = 0.639702
Epoch 27
Validation binary_cross_entropy = 1.473400
Epoch 28
Validation binary_cross_entropy = 1.866851
Epoch 29
Loss = 2.1217e+00, PNorm = 27.4607, GNorm = 20.1129, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 1.346757
Epoch 30
Validation binary_cross_entropy = 0.607099
Epoch 31
Validation binary_cross_entropy = 0.397877
Epoch 32
Validation binary_cross_entropy = 0.418558
Epoch 33
Validation binary_cross_entropy = 0.489181
Epoch 34
Loss = 1.1553e+00, PNorm = 27.4762, GNorm = 21.6203, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.603670
Epoch 35
Validation binary_cross_entropy = 0.987143
Epoch 36
Validation binary_cross_entropy = 1.226928
Epoch 37
Validation binary_cross_entropy = 1.124498
Epoch 38
Validation binary_cross_entropy = 0.739024
Epoch 39
Loss = 1.3324e+00, PNorm = 27.4871, GNorm = 22.7824, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.489980
Epoch 40
Validation binary_cross_entropy = 0.400514
Epoch 41
Validation binary_cross_entropy = 0.416216
Epoch 42
Validation binary_cross_entropy = 0.519972
Epoch 43
Validation binary_cross_entropy = 0.687678
Epoch 44
Loss = 7.8063e-01, PNorm = 27.4966, GNorm = 12.3155, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.833851
Epoch 45
Validation binary_cross_entropy = 0.928889
Epoch 46
Validation binary_cross_entropy = 0.835585
Epoch 47
Validation binary_cross_entropy = 0.754406
Epoch 48
Validation binary_cross_entropy = 0.656599
Epoch 49
Loss = 8.1308e-01, PNorm = 27.5050, GNorm = 13.7517, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.608651
Epoch 50
Validation binary_cross_entropy = 0.653038
Epoch 51
Validation binary_cross_entropy = 0.650883
Epoch 52
Validation binary_cross_entropy = 0.632492
Epoch 53
Validation binary_cross_entropy = 0.676984
Epoch 54
Loss = 8.9796e-01, PNorm = 27.5117, GNorm = 12.5756, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.807799
Epoch 55
Validation binary_cross_entropy = 0.859904
Epoch 56
Validation binary_cross_entropy = 0.873739
Epoch 57
Validation binary_cross_entropy = 0.767116
Epoch 58
Validation binary_cross_entropy = 0.705751
Epoch 59
Loss = 1.0982e+00, PNorm = 27.5198, GNorm = 13.7053, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.719345
Epoch 60
Validation binary_cross_entropy = 0.758368
Epoch 61
Validation binary_cross_entropy = 0.817039
Epoch 62
Validation binary_cross_entropy = 0.858777
Epoch 63
Validation binary_cross_entropy = 0.773736
Epoch 64
Loss = 8.5060e-01, PNorm = 27.5288, GNorm = 11.3460, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.750800
Epoch 65
Validation binary_cross_entropy = 0.768861
Epoch 66
Validation binary_cross_entropy = 0.721529
Epoch 67
Validation binary_cross_entropy = 0.635993
Epoch 68
Validation binary_cross_entropy = 0.622030
Epoch 69
Loss = 7.8359e-01, PNorm = 27.5366, GNorm = 12.0565, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.654253
Epoch 70
Validation binary_cross_entropy = 0.657883
Epoch 71
Validation binary_cross_entropy = 0.619334
Epoch 72
Validation binary_cross_entropy = 0.598731
Epoch 73
Validation binary_cross_entropy = 0.599928
Epoch 74
Loss = 7.1487e-01, PNorm = 27.5423, GNorm = 8.2698, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.602697
Epoch 75
Validation binary_cross_entropy = 0.609838
Epoch 76
Validation binary_cross_entropy = 0.625650
Epoch 77
Validation binary_cross_entropy = 0.641248
Epoch 78
Validation binary_cross_entropy = 0.659398
Epoch 79
Loss = 8.2281e-01, PNorm = 27.5479, GNorm = 13.0225, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.636860
Epoch 80
Validation binary_cross_entropy = 0.624154
Epoch 81
Validation binary_cross_entropy = 0.573482
Epoch 82
Validation binary_cross_entropy = 0.552268
Epoch 83
Validation binary_cross_entropy = 0.527541
Epoch 84
Loss = 6.9770e-01, PNorm = 27.5532, GNorm = 10.6778, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.516748
Epoch 85
Validation binary_cross_entropy = 0.555486
Epoch 86
Validation binary_cross_entropy = 0.634956
Epoch 87
Validation binary_cross_entropy = 0.687647
Epoch 88
Validation binary_cross_entropy = 0.765938
Epoch 89
Loss = 8.4210e-01, PNorm = 27.5593, GNorm = 12.6889, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.760505
Epoch 90
Validation binary_cross_entropy = 0.704553
Epoch 91
Validation binary_cross_entropy = 0.674927
Epoch 92
Validation binary_cross_entropy = 0.607528
Epoch 93
Validation binary_cross_entropy = 0.571479
Epoch 94
Loss = 7.2873e-01, PNorm = 27.5644, GNorm = 9.0271, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.564078
Epoch 95
Validation binary_cross_entropy = 0.620373
Epoch 96
Validation binary_cross_entropy = 0.717331
Epoch 97
Validation binary_cross_entropy = 0.814350
Epoch 98
Validation binary_cross_entropy = 0.846825
Epoch 99
Loss = 6.1753e-01, PNorm = 27.5702, GNorm = 8.3452, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.830863
Epoch 100
Validation binary_cross_entropy = 0.836710
Epoch 101
Validation binary_cross_entropy = 0.793910
Epoch 102
Validation binary_cross_entropy = 0.765720
Epoch 103
Validation binary_cross_entropy = 0.762702
Epoch 104
Loss = 7.6423e-01, PNorm = 27.5743, GNorm = 12.0188, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.770910
Epoch 105
Validation binary_cross_entropy = 0.763873
Epoch 106
Validation binary_cross_entropy = 0.749909
Epoch 107
Validation binary_cross_entropy = 0.696440
Epoch 108
Validation binary_cross_entropy = 0.639358
Epoch 109
Loss = 7.4716e-01, PNorm = 27.5780, GNorm = 8.3558, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.618737
Epoch 110
Validation binary_cross_entropy = 0.600331
Epoch 111
Validation binary_cross_entropy = 0.587583
Epoch 112
Validation binary_cross_entropy = 0.587074
Epoch 113
Validation binary_cross_entropy = 0.598095
Epoch 114
Loss = 7.0515e-01, PNorm = 27.5821, GNorm = 8.5716, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.607131
Epoch 115
Validation binary_cross_entropy = 0.626202
Epoch 116
Validation binary_cross_entropy = 0.633770
Epoch 117
Validation binary_cross_entropy = 0.644275
Epoch 118
Validation binary_cross_entropy = 0.617472
Epoch 119
Loss = 6.6630e-01, PNorm = 27.5862, GNorm = 13.3978, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.584663
Epoch 120
Validation binary_cross_entropy = 0.569272
Epoch 121
Validation binary_cross_entropy = 0.551310
Epoch 122
Validation binary_cross_entropy = 0.536178
Epoch 123
Validation binary_cross_entropy = 0.545578
Epoch 124
Loss = 6.5601e-01, PNorm = 27.5900, GNorm = 7.8109, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.564218
Epoch 125
Validation binary_cross_entropy = 0.605144
Epoch 126
Validation binary_cross_entropy = 0.639059
Epoch 127
Validation binary_cross_entropy = 0.680610
Epoch 128
Validation binary_cross_entropy = 0.736827
Epoch 129
Loss = 5.7727e-01, PNorm = 27.5946, GNorm = 6.5811, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.771982
Epoch 130
Validation binary_cross_entropy = 0.779774
Epoch 131
Validation binary_cross_entropy = 0.770627
Epoch 132
Validation binary_cross_entropy = 0.750881
Epoch 133
Validation binary_cross_entropy = 0.702759
Epoch 134
Loss = 4.3813e-01, PNorm = 27.5988, GNorm = 6.0060, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.658210
Epoch 135
Validation binary_cross_entropy = 0.639270
Epoch 136
Validation binary_cross_entropy = 0.637554
Epoch 137
Validation binary_cross_entropy = 0.638344
Epoch 138
Validation binary_cross_entropy = 0.657934
Epoch 139
Loss = 6.5054e-01, PNorm = 27.6029, GNorm = 7.4219, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.690257
Epoch 140
Validation binary_cross_entropy = 0.727948
Epoch 141
Validation binary_cross_entropy = 0.756497
Epoch 142
Validation binary_cross_entropy = 0.780271
Epoch 143
Validation binary_cross_entropy = 0.764196
Epoch 144
Loss = 6.8392e-01, PNorm = 27.6061, GNorm = 9.0088, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.735328
Epoch 145
Validation binary_cross_entropy = 0.715309
Epoch 146
Validation binary_cross_entropy = 0.684703
Epoch 147
Validation binary_cross_entropy = 0.659311
Epoch 148
Validation binary_cross_entropy = 0.645955
Epoch 149
Loss = 7.2632e-01, PNorm = 27.6094, GNorm = 8.9126, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.628197
Epoch 150
Validation binary_cross_entropy = 0.621437
Epoch 151
Validation binary_cross_entropy = 0.625765
Epoch 152
Validation binary_cross_entropy = 0.632862
Epoch 153
Validation binary_cross_entropy = 0.639833
Epoch 154
Loss = 5.1914e-01, PNorm = 27.6130, GNorm = 6.4318, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.643339
Epoch 155
Validation binary_cross_entropy = 0.647926
Epoch 156
Validation binary_cross_entropy = 0.645434
Epoch 157
Validation binary_cross_entropy = 0.640058
Epoch 158
Validation binary_cross_entropy = 0.620440
Epoch 159
Loss = 5.1000e-01, PNorm = 27.6176, GNorm = 7.6740, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.608252
Epoch 160
Validation binary_cross_entropy = 0.597005
Epoch 161
Validation binary_cross_entropy = 0.599468
Epoch 162
Validation binary_cross_entropy = 0.610451
Epoch 163
Validation binary_cross_entropy = 0.621136
Epoch 164
Loss = 3.6207e-01, PNorm = 27.6224, GNorm = 5.0914, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.628732
Epoch 165
Validation binary_cross_entropy = 0.632487
Epoch 166
Validation binary_cross_entropy = 0.645231
Epoch 167
Validation binary_cross_entropy = 0.657222
Epoch 168
Validation binary_cross_entropy = 0.663491
Epoch 169
Loss = 4.9054e-01, PNorm = 27.6263, GNorm = 4.5426, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.658416
Epoch 170
Validation binary_cross_entropy = 0.654902
Epoch 171
Validation binary_cross_entropy = 0.649480
Epoch 172
Validation binary_cross_entropy = 0.647051
Epoch 173
Validation binary_cross_entropy = 0.650063
Epoch 174
Loss = 7.0106e-01, PNorm = 27.6297, GNorm = 5.3384, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.653114
Epoch 175
Validation binary_cross_entropy = 0.650813
Epoch 176
Validation binary_cross_entropy = 0.639045
Epoch 177
Validation binary_cross_entropy = 0.635938
Epoch 178
Validation binary_cross_entropy = 0.639832
Epoch 179
Loss = 5.3409e-01, PNorm = 27.6336, GNorm = 5.8109, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.625469
Epoch 180
Validation binary_cross_entropy = 0.613064
Epoch 181
Validation binary_cross_entropy = 0.587778
Epoch 182
Validation binary_cross_entropy = 0.569043
Epoch 183
Validation binary_cross_entropy = 0.564083
Epoch 184
Loss = 5.0897e-01, PNorm = 27.6368, GNorm = 7.8744, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.567869
Epoch 185
Validation binary_cross_entropy = 0.582683
Epoch 186
Validation binary_cross_entropy = 0.594691
Epoch 187
Validation binary_cross_entropy = 0.598071
Epoch 188
Validation binary_cross_entropy = 0.591534
Epoch 189
Loss = 5.2362e-01, PNorm = 27.6408, GNorm = 5.3115, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.583832
Epoch 190
Validation binary_cross_entropy = 0.584451
Epoch 191
Validation binary_cross_entropy = 0.584558
Epoch 192
Validation binary_cross_entropy = 0.581924
Epoch 193
Validation binary_cross_entropy = 0.572786
Epoch 194
Loss = 5.6678e-01, PNorm = 27.6457, GNorm = 7.6985, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.569314
Epoch 195
Validation binary_cross_entropy = 0.562365
Epoch 196
Validation binary_cross_entropy = 0.558474
Epoch 197
Validation binary_cross_entropy = 0.564730
Epoch 198
Validation binary_cross_entropy = 0.576345
Epoch 199
Loss = 4.9981e-01, PNorm = 27.6494, GNorm = 6.4702, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.597659
Epoch 200
Validation binary_cross_entropy = 0.620281
Epoch 201
Validation binary_cross_entropy = 0.633141
Epoch 202
Validation binary_cross_entropy = 0.634623
Epoch 203
Validation binary_cross_entropy = 0.621872
Epoch 204
Loss = 5.3044e-01, PNorm = 27.6531, GNorm = 6.4319, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.611626
Epoch 205
Validation binary_cross_entropy = 0.602749
Epoch 206
Validation binary_cross_entropy = 0.593233
Epoch 207
Validation binary_cross_entropy = 0.587132
Epoch 208
Validation binary_cross_entropy = 0.584231
Epoch 209
Loss = 6.5396e-01, PNorm = 27.6557, GNorm = 5.8873, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.588107
Epoch 210
Validation binary_cross_entropy = 0.598507
Epoch 211
Validation binary_cross_entropy = 0.600802
Epoch 212
Validation binary_cross_entropy = 0.606633
Epoch 213
Validation binary_cross_entropy = 0.614985
Epoch 214
Loss = 5.0704e-01, PNorm = 27.6594, GNorm = 6.1408, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.620726
Epoch 215
Validation binary_cross_entropy = 0.631155
Epoch 216
Validation binary_cross_entropy = 0.632647
Epoch 217
Validation binary_cross_entropy = 0.629105
Epoch 218
Validation binary_cross_entropy = 0.625333
Epoch 219
Loss = 5.5897e-01, PNorm = 27.6644, GNorm = 4.9220, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.621604
Epoch 220
Validation binary_cross_entropy = 0.610989
Epoch 221
Validation binary_cross_entropy = 0.596619
Epoch 222
Validation binary_cross_entropy = 0.587406
Epoch 223
Validation binary_cross_entropy = 0.590930
Epoch 224
Loss = 4.5336e-01, PNorm = 27.6685, GNorm = 4.6499, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.592576
Epoch 225
Validation binary_cross_entropy = 0.594567
Epoch 226
Validation binary_cross_entropy = 0.594807
Epoch 227
Validation binary_cross_entropy = 0.589211
Epoch 228
Validation binary_cross_entropy = 0.576941
Epoch 229
Loss = 5.3244e-01, PNorm = 27.6724, GNorm = 7.7130, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.568823
Epoch 230
Validation binary_cross_entropy = 0.573477
Epoch 231
Validation binary_cross_entropy = 0.575886
Epoch 232
Validation binary_cross_entropy = 0.580473
Epoch 233
Validation binary_cross_entropy = 0.586943
Epoch 234
Loss = 4.6306e-01, PNorm = 27.6766, GNorm = 8.9734, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.602796
Epoch 235
Validation binary_cross_entropy = 0.620849
Epoch 236
Validation binary_cross_entropy = 0.628734
Epoch 237
Validation binary_cross_entropy = 0.633702
Epoch 238
Validation binary_cross_entropy = 0.628008
Epoch 239
Loss = 4.1058e-01, PNorm = 27.6795, GNorm = 5.3715, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.617139
Epoch 240
Validation binary_cross_entropy = 0.606160
Epoch 241
Validation binary_cross_entropy = 0.604926
Epoch 242
Validation binary_cross_entropy = 0.610419
Epoch 243
Validation binary_cross_entropy = 0.620155
Epoch 244
Loss = 4.3054e-01, PNorm = 27.6831, GNorm = 5.0418, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.621512
Epoch 245
Validation binary_cross_entropy = 0.630480
Epoch 246
Validation binary_cross_entropy = 0.644646
Epoch 247
Validation binary_cross_entropy = 0.669844
Epoch 248
Validation binary_cross_entropy = 0.683337
Epoch 249
Loss = 5.5828e-01, PNorm = 27.6869, GNorm = 5.0842, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.691189
Epoch 250
Validation binary_cross_entropy = 0.693110
Epoch 251
Validation binary_cross_entropy = 0.708543
Epoch 252
Validation binary_cross_entropy = 0.713705
Epoch 253
Validation binary_cross_entropy = 0.694124
Epoch 254
Loss = 5.3897e-01, PNorm = 27.6905, GNorm = 6.3347, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.666607
Epoch 255
Validation binary_cross_entropy = 0.632986
Epoch 256
Validation binary_cross_entropy = 0.611970
Epoch 257
Validation binary_cross_entropy = 0.611780
Epoch 258
Validation binary_cross_entropy = 0.610561
Epoch 259
Loss = 4.8138e-01, PNorm = 27.6951, GNorm = 4.6785, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.615910
Epoch 260
Validation binary_cross_entropy = 0.620182
Epoch 261
Validation binary_cross_entropy = 0.625840
Epoch 262
Validation binary_cross_entropy = 0.636187
Epoch 263
Validation binary_cross_entropy = 0.645054
Epoch 264
Loss = 4.4609e-01, PNorm = 27.6997, GNorm = 6.0677, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.668204
Epoch 265
Validation binary_cross_entropy = 0.684922
Epoch 266
Validation binary_cross_entropy = 0.692966
Epoch 267
Validation binary_cross_entropy = 0.684569
Epoch 268
Validation binary_cross_entropy = 0.666247
Epoch 269
Loss = 3.6309e-01, PNorm = 27.7026, GNorm = 3.7255, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.642784
Epoch 270
Validation binary_cross_entropy = 0.618028
Epoch 271
Validation binary_cross_entropy = 0.602350
Epoch 272
Validation binary_cross_entropy = 0.600818
Epoch 273
Validation binary_cross_entropy = 0.609247
Epoch 274
Loss = 4.0642e-01, PNorm = 27.7054, GNorm = 5.8767, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.619003
Epoch 275
Validation binary_cross_entropy = 0.636302
Epoch 276
Validation binary_cross_entropy = 0.647695
Epoch 277
Validation binary_cross_entropy = 0.655340
Epoch 278
Validation binary_cross_entropy = 0.656891
Epoch 279
Loss = 4.9891e-01, PNorm = 27.7089, GNorm = 3.8703, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.655055
Epoch 280
Validation binary_cross_entropy = 0.645150
Epoch 281
Validation binary_cross_entropy = 0.628507
Epoch 282
Validation binary_cross_entropy = 0.612895
Epoch 283
Validation binary_cross_entropy = 0.602025
Epoch 284
Loss = 4.9706e-01, PNorm = 27.7125, GNorm = 4.6708, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.591253
Epoch 285
Validation binary_cross_entropy = 0.583788
Epoch 286
Validation binary_cross_entropy = 0.582944
Epoch 287
Validation binary_cross_entropy = 0.584718
Epoch 288
Validation binary_cross_entropy = 0.584893
Epoch 289
Loss = 4.7856e-01, PNorm = 27.7163, GNorm = 5.5111, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.585977
Epoch 290
Validation binary_cross_entropy = 0.594319
Epoch 291
Validation binary_cross_entropy = 0.606633
Epoch 292
Validation binary_cross_entropy = 0.617331
Epoch 293
Validation binary_cross_entropy = 0.620004
Epoch 294
Loss = 5.4677e-01, PNorm = 27.7193, GNorm = 4.6235, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.617997
Epoch 295
Validation binary_cross_entropy = 0.609569
Epoch 296
Validation binary_cross_entropy = 0.600142
Epoch 297
Validation binary_cross_entropy = 0.592509
Epoch 298
Validation binary_cross_entropy = 0.585498
Epoch 299
Loss = 5.0149e-01, PNorm = 27.7225, GNorm = 6.7376, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.581785
Model 0 best validation binary_cross_entropy = 0.384625 on epoch 25
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.298599
Ensemble test binary_cross_entropy = 0.298599
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 281,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 2.717009
Epoch 1
Validation binary_cross_entropy = 3.497844
Epoch 2
Validation binary_cross_entropy = 0.977954
Epoch 3
Validation binary_cross_entropy = 0.350902
Epoch 4
Loss = 4.0747e+00, PNorm = 27.3560, GNorm = 43.1475, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.452250
Epoch 5
Validation binary_cross_entropy = 0.377791
Epoch 6
Validation binary_cross_entropy = 0.869348
Epoch 7
Validation binary_cross_entropy = 0.520647
Epoch 8
Validation binary_cross_entropy = 0.455869
Epoch 9
Loss = 3.8490e+00, PNorm = 27.3743, GNorm = 44.5412, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.462395
Epoch 10
Validation binary_cross_entropy = 0.934636
Epoch 11
Validation binary_cross_entropy = 1.031504
Epoch 12
Validation binary_cross_entropy = 0.642392
Epoch 13
Validation binary_cross_entropy = 0.543185
Epoch 14
Loss = 2.7885e+00, PNorm = 27.4026, GNorm = 33.5968, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.534687
Epoch 15
Validation binary_cross_entropy = 0.667628
Epoch 16
Validation binary_cross_entropy = 0.969357
Epoch 17
Validation binary_cross_entropy = 0.522614
Epoch 18
Validation binary_cross_entropy = 0.460593
Epoch 19
Loss = 2.2877e+00, PNorm = 27.4266, GNorm = 30.8798, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.456227
Epoch 20
Validation binary_cross_entropy = 0.555462
Epoch 21
Validation binary_cross_entropy = 1.522747
Epoch 22
Validation binary_cross_entropy = 1.646387
Epoch 23
Validation binary_cross_entropy = 1.077841
Epoch 24
Loss = 2.5631e+00, PNorm = 27.4460, GNorm = 25.5872, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.590655
Epoch 25
Validation binary_cross_entropy = 0.434191
Epoch 26
Validation binary_cross_entropy = 0.435771
Epoch 27
Validation binary_cross_entropy = 0.523120
Epoch 28
Validation binary_cross_entropy = 0.856900
Epoch 29
Loss = 1.3226e+00, PNorm = 27.4605, GNorm = 21.4057, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.780119
Epoch 30
Validation binary_cross_entropy = 0.680165
Epoch 31
Validation binary_cross_entropy = 0.620852
Epoch 32
Validation binary_cross_entropy = 0.672809
Epoch 33
Validation binary_cross_entropy = 0.652636
Epoch 34
Loss = 1.0548e+00, PNorm = 27.4743, GNorm = 16.5805, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.621248
Epoch 35
Validation binary_cross_entropy = 0.505370
Epoch 36
Validation binary_cross_entropy = 0.413760
Epoch 37
Validation binary_cross_entropy = 0.387713
Epoch 38
Validation binary_cross_entropy = 0.458242
Epoch 39
Loss = 1.3946e+00, PNorm = 27.4861, GNorm = 15.5257, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.613510
Epoch 40
Validation binary_cross_entropy = 0.749581
Epoch 41
Validation binary_cross_entropy = 0.794451
Epoch 42
Validation binary_cross_entropy = 0.809743
Epoch 43
Validation binary_cross_entropy = 0.794519
Epoch 44
Loss = 1.4053e+00, PNorm = 27.4958, GNorm = 12.6801, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.599385
Epoch 45
Validation binary_cross_entropy = 0.500366
Epoch 46
Validation binary_cross_entropy = 0.419906
Epoch 47
Validation binary_cross_entropy = 0.436553
Epoch 48
Validation binary_cross_entropy = 0.454423
Epoch 49
Loss = 9.7279e-01, PNorm = 27.5052, GNorm = 12.6579, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.497670
Epoch 50
Validation binary_cross_entropy = 0.631847
Epoch 51
Validation binary_cross_entropy = 0.755419
Epoch 52
Validation binary_cross_entropy = 0.644149
Epoch 53
Validation binary_cross_entropy = 0.488304
Epoch 54
Loss = 1.1627e+00, PNorm = 27.5143, GNorm = 18.6200, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.407098
Epoch 55
Validation binary_cross_entropy = 0.384150
Epoch 56
Validation binary_cross_entropy = 0.407871
Epoch 57
Validation binary_cross_entropy = 0.472646
Epoch 58
Validation binary_cross_entropy = 0.590012
Epoch 59
Loss = 8.3298e-01, PNorm = 27.5222, GNorm = 9.9527, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.666259
Epoch 60
Validation binary_cross_entropy = 0.679175
Epoch 61
Validation binary_cross_entropy = 0.668306
Epoch 62
Validation binary_cross_entropy = 0.597482
Epoch 63
Validation binary_cross_entropy = 0.537455
Epoch 64
Loss = 1.3039e+00, PNorm = 27.5302, GNorm = 14.6385, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.482243
Epoch 65
Validation binary_cross_entropy = 0.433287
Epoch 66
Validation binary_cross_entropy = 0.417015
Epoch 67
Validation binary_cross_entropy = 0.424589
Epoch 68
Validation binary_cross_entropy = 0.470624
Epoch 69
Loss = 8.6970e-01, PNorm = 27.5368, GNorm = 14.7673, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.540275
Epoch 70
Validation binary_cross_entropy = 0.645767
Epoch 71
Validation binary_cross_entropy = 0.704303
Epoch 72
Validation binary_cross_entropy = 0.683745
Epoch 73
Validation binary_cross_entropy = 0.551270
Epoch 74
Loss = 7.6901e-01, PNorm = 27.5436, GNorm = 9.6261, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.461198
Epoch 75
Validation binary_cross_entropy = 0.389784
Epoch 76
Validation binary_cross_entropy = 0.375901
Epoch 77
Validation binary_cross_entropy = 0.374218
Epoch 78
Validation binary_cross_entropy = 0.391011
Epoch 79
Loss = 1.0210e+00, PNorm = 27.5506, GNorm = 12.3346, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.445896
Epoch 80
Validation binary_cross_entropy = 0.527095
Epoch 81
Validation binary_cross_entropy = 0.587126
Epoch 82
Validation binary_cross_entropy = 0.597749
Epoch 83
Validation binary_cross_entropy = 0.593436
Epoch 84
Loss = 7.1349e-01, PNorm = 27.5572, GNorm = 10.1341, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.555300
Epoch 85
Validation binary_cross_entropy = 0.508636
Epoch 86
Validation binary_cross_entropy = 0.490569
Epoch 87
Validation binary_cross_entropy = 0.504964
Epoch 88
Validation binary_cross_entropy = 0.518902
Epoch 89
Loss = 1.0399e+00, PNorm = 27.5615, GNorm = 12.9049, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.534126
Epoch 90
Validation binary_cross_entropy = 0.587168
Epoch 91
Validation binary_cross_entropy = 0.618537
Epoch 92
Validation binary_cross_entropy = 0.625679
Epoch 93
Validation binary_cross_entropy = 0.627412
Epoch 94
Loss = 1.2213e+00, PNorm = 27.5662, GNorm = 17.3534, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.612144
Epoch 95
Validation binary_cross_entropy = 0.574420
Epoch 96
Validation binary_cross_entropy = 0.540319
Epoch 97
Validation binary_cross_entropy = 0.502975
Epoch 98
Validation binary_cross_entropy = 0.492192
Epoch 99
Loss = 8.0912e-01, PNorm = 27.5709, GNorm = 15.5660, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.497200
Epoch 100
Validation binary_cross_entropy = 0.509726
Epoch 101
Validation binary_cross_entropy = 0.526524
Epoch 102
Validation binary_cross_entropy = 0.546720
Epoch 103
Validation binary_cross_entropy = 0.552979
Epoch 104
Loss = 6.4650e-01, PNorm = 27.5766, GNorm = 8.8167, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.567245
Epoch 105
Validation binary_cross_entropy = 0.565329
Epoch 106
Validation binary_cross_entropy = 0.544355
Epoch 107
Validation binary_cross_entropy = 0.511275
Epoch 108
Validation binary_cross_entropy = 0.494111
Epoch 109
Loss = 5.5879e-01, PNorm = 27.5822, GNorm = 6.8589, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.497558
Epoch 110
Validation binary_cross_entropy = 0.512329
Epoch 111
Validation binary_cross_entropy = 0.537512
Epoch 112
Validation binary_cross_entropy = 0.561260
Epoch 113
Validation binary_cross_entropy = 0.578012
Epoch 114
Loss = 7.0172e-01, PNorm = 27.5872, GNorm = 7.8631, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.591197
Epoch 115
Validation binary_cross_entropy = 0.583399
Epoch 116
Validation binary_cross_entropy = 0.568452
Epoch 117
Validation binary_cross_entropy = 0.559908
Epoch 118
Validation binary_cross_entropy = 0.561259
Epoch 119
Loss = 4.8312e-01, PNorm = 27.5914, GNorm = 5.2868, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.579677
Epoch 120
Validation binary_cross_entropy = 0.580056
Epoch 121
Validation binary_cross_entropy = 0.584290
Epoch 122
Validation binary_cross_entropy = 0.583488
Epoch 123
Validation binary_cross_entropy = 0.595552
Epoch 124
Loss = 7.8273e-01, PNorm = 27.5952, GNorm = 10.2034, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.602084
Epoch 125
Validation binary_cross_entropy = 0.601021
Epoch 126
Validation binary_cross_entropy = 0.589002
Epoch 127
Validation binary_cross_entropy = 0.573260
Epoch 128
Validation binary_cross_entropy = 0.568875
Epoch 129
Loss = 7.9923e-01, PNorm = 27.5992, GNorm = 9.5212, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.558303
Epoch 130
Validation binary_cross_entropy = 0.540481
Epoch 131
Validation binary_cross_entropy = 0.537614
Epoch 132
Validation binary_cross_entropy = 0.549064
Epoch 133
Validation binary_cross_entropy = 0.577862
Epoch 134
Loss = 5.7290e-01, PNorm = 27.6041, GNorm = 5.9518, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.592528
Epoch 135
Validation binary_cross_entropy = 0.591990
Epoch 136
Validation binary_cross_entropy = 0.589615
Epoch 137
Validation binary_cross_entropy = 0.581689
Epoch 138
Validation binary_cross_entropy = 0.586252
Epoch 139
Loss = 6.8424e-01, PNorm = 27.6086, GNorm = 8.1969, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.578793
Epoch 140
Validation binary_cross_entropy = 0.552965
Epoch 141
Validation binary_cross_entropy = 0.531629
Epoch 142
Validation binary_cross_entropy = 0.522307
Epoch 143
Validation binary_cross_entropy = 0.515557
Epoch 144
Loss = 4.2618e-01, PNorm = 27.6132, GNorm = 5.3636, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.518062
Epoch 145
Validation binary_cross_entropy = 0.524444
Epoch 146
Validation binary_cross_entropy = 0.538546
Epoch 147
Validation binary_cross_entropy = 0.540645
Epoch 148
Validation binary_cross_entropy = 0.547515
Epoch 149
Loss = 7.6497e-01, PNorm = 27.6182, GNorm = 8.6480, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.563725
Epoch 150
Validation binary_cross_entropy = 0.579160
Epoch 151
Validation binary_cross_entropy = 0.585585
Epoch 152
Validation binary_cross_entropy = 0.565400
Epoch 153
Validation binary_cross_entropy = 0.548805
Epoch 154
Loss = 5.6514e-01, PNorm = 27.6230, GNorm = 5.7646, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.548675
Epoch 155
Validation binary_cross_entropy = 0.549775
Epoch 156
Validation binary_cross_entropy = 0.542262
Epoch 157
Validation binary_cross_entropy = 0.524241
Epoch 158
Validation binary_cross_entropy = 0.499603
Epoch 159
Loss = 4.9470e-01, PNorm = 27.6279, GNorm = 6.3920, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.477395
Epoch 160
Validation binary_cross_entropy = 0.474450
Epoch 161
Validation binary_cross_entropy = 0.480594
Epoch 162
Validation binary_cross_entropy = 0.495723
Epoch 163
Validation binary_cross_entropy = 0.509257
Epoch 164
Loss = 5.1002e-01, PNorm = 27.6324, GNorm = 6.8176, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.525857
Epoch 165
Validation binary_cross_entropy = 0.547894
Epoch 166
Validation binary_cross_entropy = 0.550667
Epoch 167
Validation binary_cross_entropy = 0.541078
Epoch 168
Validation binary_cross_entropy = 0.534549
Epoch 169
Loss = 7.1637e-01, PNorm = 27.6368, GNorm = 6.1233, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.531329
Epoch 170
Validation binary_cross_entropy = 0.527525
Epoch 171
Validation binary_cross_entropy = 0.525389
Epoch 172
Validation binary_cross_entropy = 0.530000
Epoch 173
Validation binary_cross_entropy = 0.536693
Epoch 174
Loss = 5.4219e-01, PNorm = 27.6415, GNorm = 6.1676, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.543184
Epoch 175
Validation binary_cross_entropy = 0.552255
Epoch 176
Validation binary_cross_entropy = 0.559709
Epoch 177
Validation binary_cross_entropy = 0.569613
Epoch 178
Validation binary_cross_entropy = 0.583322
Epoch 179
Loss = 5.9899e-01, PNorm = 27.6465, GNorm = 7.8394, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.597646
Epoch 180
Validation binary_cross_entropy = 0.608134
Epoch 181
Validation binary_cross_entropy = 0.610959
Epoch 182
Validation binary_cross_entropy = 0.602956
Epoch 183
Validation binary_cross_entropy = 0.590359
Epoch 184
Loss = 5.4441e-01, PNorm = 27.6506, GNorm = 6.9009, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.580913
Epoch 185
Validation binary_cross_entropy = 0.570387
Epoch 186
Validation binary_cross_entropy = 0.564568
Epoch 187
Validation binary_cross_entropy = 0.562225
Epoch 188
Validation binary_cross_entropy = 0.563766
Epoch 189
Loss = 4.8423e-01, PNorm = 27.6547, GNorm = 4.3465, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.566564
Epoch 190
Validation binary_cross_entropy = 0.568696
Epoch 191
Validation binary_cross_entropy = 0.580139
Epoch 192
Validation binary_cross_entropy = 0.595732
Epoch 193
Validation binary_cross_entropy = 0.600210
Epoch 194
Loss = 5.8773e-01, PNorm = 27.6599, GNorm = 5.8937, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.600479
Epoch 195
Validation binary_cross_entropy = 0.599180
Epoch 196
Validation binary_cross_entropy = 0.592721
Epoch 197
Validation binary_cross_entropy = 0.583905
Epoch 198
Validation binary_cross_entropy = 0.587685
Epoch 199
Loss = 5.0898e-01, PNorm = 27.6645, GNorm = 4.8683, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.601982
Epoch 200
Validation binary_cross_entropy = 0.612339
Epoch 201
Validation binary_cross_entropy = 0.614496
Epoch 202
Validation binary_cross_entropy = 0.608597
Epoch 203
Validation binary_cross_entropy = 0.605122
Epoch 204
Loss = 4.9551e-01, PNorm = 27.6687, GNorm = 4.6980, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.600230
Epoch 205
Validation binary_cross_entropy = 0.596421
Epoch 206
Validation binary_cross_entropy = 0.593073
Epoch 207
Validation binary_cross_entropy = 0.598346
Epoch 208
Validation binary_cross_entropy = 0.610171
Epoch 209
Loss = 4.9564e-01, PNorm = 27.6726, GNorm = 4.4514, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.621429
Epoch 210
Validation binary_cross_entropy = 0.632875
Epoch 211
Validation binary_cross_entropy = 0.639627
Epoch 212
Validation binary_cross_entropy = 0.642358
Epoch 213
Validation binary_cross_entropy = 0.630122
Epoch 214
Loss = 5.3012e-01, PNorm = 27.6767, GNorm = 4.4998, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.620636
Epoch 215
Validation binary_cross_entropy = 0.600940
Epoch 216
Validation binary_cross_entropy = 0.576834
Epoch 217
Validation binary_cross_entropy = 0.560631
Epoch 218
Validation binary_cross_entropy = 0.554861
Epoch 219
Loss = 5.3405e-01, PNorm = 27.6804, GNorm = 5.1290, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.550662
Epoch 220
Validation binary_cross_entropy = 0.543641
Epoch 221
Validation binary_cross_entropy = 0.539657
Epoch 222
Validation binary_cross_entropy = 0.539927
Epoch 223
Validation binary_cross_entropy = 0.545783
Epoch 224
Loss = 4.4260e-01, PNorm = 27.6842, GNorm = 4.5005, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.561231
Epoch 225
Validation binary_cross_entropy = 0.578392
Epoch 226
Validation binary_cross_entropy = 0.591842
Epoch 227
Validation binary_cross_entropy = 0.594664
Epoch 228
Validation binary_cross_entropy = 0.596473
Epoch 229
Loss = 4.3982e-01, PNorm = 27.6882, GNorm = 4.5713, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.599338
Epoch 230
Validation binary_cross_entropy = 0.599871
Epoch 231
Validation binary_cross_entropy = 0.592350
Epoch 232
Validation binary_cross_entropy = 0.583788
Epoch 233
Validation binary_cross_entropy = 0.581701
Epoch 234
Loss = 5.9545e-01, PNorm = 27.6916, GNorm = 3.7157, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.574832
Epoch 235
Validation binary_cross_entropy = 0.564012
Epoch 236
Validation binary_cross_entropy = 0.554895
Epoch 237
Validation binary_cross_entropy = 0.553495
Epoch 238
Validation binary_cross_entropy = 0.562367
Epoch 239
Loss = 5.8244e-01, PNorm = 27.6954, GNorm = 6.3789, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.577106
Epoch 240
Validation binary_cross_entropy = 0.585295
Epoch 241
Validation binary_cross_entropy = 0.595279
Epoch 242
Validation binary_cross_entropy = 0.601321
Epoch 243
Validation binary_cross_entropy = 0.604631
Epoch 244
Loss = 5.9596e-01, PNorm = 27.6999, GNorm = 5.8347, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.606093
Epoch 245
Validation binary_cross_entropy = 0.598389
Epoch 246
Validation binary_cross_entropy = 0.586346
Epoch 247
Validation binary_cross_entropy = 0.573470
Epoch 248
Validation binary_cross_entropy = 0.555577
Epoch 249
Loss = 4.6848e-01, PNorm = 27.7040, GNorm = 6.0625, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.542432
Epoch 250
Validation binary_cross_entropy = 0.532883
Epoch 251
Validation binary_cross_entropy = 0.522879
Epoch 252
Validation binary_cross_entropy = 0.520805
Epoch 253
Validation binary_cross_entropy = 0.527626
Epoch 254
Loss = 5.0682e-01, PNorm = 27.7080, GNorm = 3.6143, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.537301
Epoch 255
Validation binary_cross_entropy = 0.541020
Epoch 256
Validation binary_cross_entropy = 0.542863
Epoch 257
Validation binary_cross_entropy = 0.547422
Epoch 258
Validation binary_cross_entropy = 0.547588
Epoch 259
Loss = 3.6975e-01, PNorm = 27.7126, GNorm = 4.4244, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.549802
Epoch 260
Validation binary_cross_entropy = 0.549890
Epoch 261
Validation binary_cross_entropy = 0.548804
Epoch 262
Validation binary_cross_entropy = 0.549810
Epoch 263
Validation binary_cross_entropy = 0.554959
Epoch 264
Loss = 4.9617e-01, PNorm = 27.7172, GNorm = 5.0262, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.561104
Epoch 265
Validation binary_cross_entropy = 0.574155
Epoch 266
Validation binary_cross_entropy = 0.585480
Epoch 267
Validation binary_cross_entropy = 0.600450
Epoch 268
Validation binary_cross_entropy = 0.620966
Epoch 269
Loss = 4.4042e-01, PNorm = 27.7223, GNorm = 7.5739, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.628542
Epoch 270
Validation binary_cross_entropy = 0.619867
Epoch 271
Validation binary_cross_entropy = 0.616220
Epoch 272
Validation binary_cross_entropy = 0.613807
Epoch 273
Validation binary_cross_entropy = 0.612636
Epoch 274
Loss = 3.6456e-01, PNorm = 27.7268, GNorm = 5.3714, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.610957
Epoch 275
Validation binary_cross_entropy = 0.607992
Epoch 276
Validation binary_cross_entropy = 0.605913
Epoch 277
Validation binary_cross_entropy = 0.608197
Epoch 278
Validation binary_cross_entropy = 0.608243
Epoch 279
Loss = 5.0531e-01, PNorm = 27.7303, GNorm = 5.7686, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.609890
Epoch 280
Validation binary_cross_entropy = 0.606272
Epoch 281
Validation binary_cross_entropy = 0.606771
Epoch 282
Validation binary_cross_entropy = 0.606590
Epoch 283
Validation binary_cross_entropy = 0.607150
Epoch 284
Loss = 4.5683e-01, PNorm = 27.7339, GNorm = 3.2578, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.603835
Epoch 285
Validation binary_cross_entropy = 0.604236
Epoch 286
Validation binary_cross_entropy = 0.604941
Epoch 287
Validation binary_cross_entropy = 0.594172
Epoch 288
Validation binary_cross_entropy = 0.587350
Epoch 289
Loss = 4.1832e-01, PNorm = 27.7384, GNorm = 3.6310, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.576526
Epoch 290
Validation binary_cross_entropy = 0.569535
Epoch 291
Validation binary_cross_entropy = 0.560491
Epoch 292
Validation binary_cross_entropy = 0.553880
Epoch 293
Validation binary_cross_entropy = 0.546937
Epoch 294
Loss = 5.5142e-01, PNorm = 27.7424, GNorm = 4.7435, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.540945
Epoch 295
Validation binary_cross_entropy = 0.541471
Epoch 296
Validation binary_cross_entropy = 0.539293
Epoch 297
Validation binary_cross_entropy = 0.540612
Epoch 298
Validation binary_cross_entropy = 0.548157
Epoch 299
Loss = 4.6029e-01, PNorm = 27.7464, GNorm = 5.1300, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.562332
Model 0 best validation binary_cross_entropy = 0.350902 on epoch 3
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.328809
Ensemble test binary_cross_entropy = 0.328809
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 281,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.468023
Epoch 1
Validation binary_cross_entropy = 1.133833
Epoch 2
Validation binary_cross_entropy = 1.639620
Epoch 3
Validation binary_cross_entropy = 0.727288
Epoch 4
Loss = 4.3975e+00, PNorm = 27.3569, GNorm = 55.0432, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.454252
Epoch 5
Validation binary_cross_entropy = 0.491739
Epoch 6
Validation binary_cross_entropy = 0.608321
Epoch 7
Validation binary_cross_entropy = 1.456760
Epoch 8
Validation binary_cross_entropy = 2.278124
Epoch 9
Loss = 3.2245e+00, PNorm = 27.3733, GNorm = 30.7303, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 1.866544
Epoch 10
Validation binary_cross_entropy = 0.724817
Epoch 11
Validation binary_cross_entropy = 0.601557
Epoch 12
Validation binary_cross_entropy = 0.952576
Epoch 13
Validation binary_cross_entropy = 1.459973
Epoch 14
Loss = 1.6268e+00, PNorm = 27.3991, GNorm = 27.2175, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.691335
Epoch 15
Validation binary_cross_entropy = 0.491255
Epoch 16
Validation binary_cross_entropy = 0.574273
Epoch 17
Validation binary_cross_entropy = 0.729629
Epoch 18
Validation binary_cross_entropy = 1.107679
Epoch 19
Loss = 2.0961e+00, PNorm = 27.4243, GNorm = 32.4673, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 1.126519
Epoch 20
Validation binary_cross_entropy = 0.424232
Epoch 21
Validation binary_cross_entropy = 0.328180
Epoch 22
Validation binary_cross_entropy = 0.329989
Epoch 23
Validation binary_cross_entropy = 0.408602
Epoch 24
Loss = 1.4138e+00, PNorm = 27.4436, GNorm = 18.8530, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 1.026761
Epoch 25
Validation binary_cross_entropy = 1.388089
Epoch 26
Validation binary_cross_entropy = 0.848279
Epoch 27
Validation binary_cross_entropy = 0.402739
Epoch 28
Validation binary_cross_entropy = 0.377265
Epoch 29
Loss = 1.3162e+00, PNorm = 27.4594, GNorm = 25.3964, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.396548
Epoch 30
Validation binary_cross_entropy = 0.416002
Epoch 31
Validation binary_cross_entropy = 0.398415
Epoch 32
Validation binary_cross_entropy = 0.425970
Epoch 33
Validation binary_cross_entropy = 0.390207
Epoch 34
Loss = 1.4514e+00, PNorm = 27.4724, GNorm = 20.2785, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.403104
Epoch 35
Validation binary_cross_entropy = 0.507067
Epoch 36
Validation binary_cross_entropy = 0.549811
Epoch 37
Validation binary_cross_entropy = 0.562102
Epoch 38
Validation binary_cross_entropy = 0.504374
Epoch 39
Loss = 1.5767e+00, PNorm = 27.4846, GNorm = 23.0422, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.529801
Epoch 40
Validation binary_cross_entropy = 0.658596
Epoch 41
Validation binary_cross_entropy = 1.034796
Epoch 42
Validation binary_cross_entropy = 1.155054
Epoch 43
Validation binary_cross_entropy = 0.955376
Epoch 44
Loss = 1.0724e+00, PNorm = 27.4958, GNorm = 16.7523, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.643407
Epoch 45
Validation binary_cross_entropy = 0.512242
Epoch 46
Validation binary_cross_entropy = 0.520622
Epoch 47
Validation binary_cross_entropy = 0.613307
Epoch 48
Validation binary_cross_entropy = 0.770331
Epoch 49
Loss = 8.9978e-01, PNorm = 27.5070, GNorm = 15.1062, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.904721
Epoch 50
Validation binary_cross_entropy = 0.956879
Epoch 51
Validation binary_cross_entropy = 0.928515
Epoch 52
Validation binary_cross_entropy = 0.844119
Epoch 53
Validation binary_cross_entropy = 0.703850
Epoch 54
Loss = 1.0118e+00, PNorm = 27.5150, GNorm = 12.2509, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.705167
Epoch 55
Validation binary_cross_entropy = 0.672016
Epoch 56
Validation binary_cross_entropy = 0.707319
Epoch 57
Validation binary_cross_entropy = 0.709018
Epoch 58
Validation binary_cross_entropy = 0.761184
Epoch 59
Loss = 8.4329e-01, PNorm = 27.5217, GNorm = 12.3762, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.868781
Epoch 60
Validation binary_cross_entropy = 0.885731
Epoch 61
Validation binary_cross_entropy = 0.816601
Epoch 62
Validation binary_cross_entropy = 0.684302
Epoch 63
Validation binary_cross_entropy = 0.553871
Epoch 64
Loss = 1.1479e+00, PNorm = 27.5288, GNorm = 13.5516, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.479775
Epoch 65
Validation binary_cross_entropy = 0.464446
Epoch 66
Validation binary_cross_entropy = 0.492245
Epoch 67
Validation binary_cross_entropy = 0.558606
Epoch 68
Validation binary_cross_entropy = 0.610195
Epoch 69
Loss = 6.8381e-01, PNorm = 27.5360, GNorm = 11.0938, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.654903
Epoch 70
Validation binary_cross_entropy = 0.663720
Epoch 71
Validation binary_cross_entropy = 0.678155
Epoch 72
Validation binary_cross_entropy = 0.635091
Epoch 73
Validation binary_cross_entropy = 0.589638
Epoch 74
Loss = 6.0148e-01, PNorm = 27.5432, GNorm = 8.0068, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.546405
Epoch 75
Validation binary_cross_entropy = 0.529966
Epoch 76
Validation binary_cross_entropy = 0.530524
Epoch 77
Validation binary_cross_entropy = 0.520802
Epoch 78
Validation binary_cross_entropy = 0.528186
Epoch 79
Loss = 9.8362e-01, PNorm = 27.5498, GNorm = 12.7635, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.570515
Epoch 80
Validation binary_cross_entropy = 0.639321
Epoch 81
Validation binary_cross_entropy = 0.648053
Epoch 82
Validation binary_cross_entropy = 0.642724
Epoch 83
Validation binary_cross_entropy = 0.632798
Epoch 84
Loss = 6.6459e-01, PNorm = 27.5550, GNorm = 8.5508, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.636552
Epoch 85
Validation binary_cross_entropy = 0.657219
Epoch 86
Validation binary_cross_entropy = 0.657490
Epoch 87
Validation binary_cross_entropy = 0.663092
Epoch 88
Validation binary_cross_entropy = 0.690436
Epoch 89
Loss = 8.5947e-01, PNorm = 27.5605, GNorm = 12.9939, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.718436
Epoch 90
Validation binary_cross_entropy = 0.716641
Epoch 91
Validation binary_cross_entropy = 0.689066
Epoch 92
Validation binary_cross_entropy = 0.667975
Epoch 93
Validation binary_cross_entropy = 0.677716
Epoch 94
Loss = 8.5711e-01, PNorm = 27.5678, GNorm = 10.8215, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.689442
Epoch 95
Validation binary_cross_entropy = 0.677346
Epoch 96
Validation binary_cross_entropy = 0.624800
Epoch 97
Validation binary_cross_entropy = 0.581490
Epoch 98
Validation binary_cross_entropy = 0.577637
Epoch 99
Loss = 7.4895e-01, PNorm = 27.5737, GNorm = 9.6340, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.580462
Epoch 100
Validation binary_cross_entropy = 0.610178
Epoch 101
Validation binary_cross_entropy = 0.642170
Epoch 102
Validation binary_cross_entropy = 0.667307
Epoch 103
Validation binary_cross_entropy = 0.692876
Epoch 104
Loss = 5.5120e-01, PNorm = 27.5784, GNorm = 6.5122, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.701415
Epoch 105
Validation binary_cross_entropy = 0.664727
Epoch 106
Validation binary_cross_entropy = 0.583475
Epoch 107
Validation binary_cross_entropy = 0.514322
Epoch 108
Validation binary_cross_entropy = 0.462167
Epoch 109
Loss = 6.2401e-01, PNorm = 27.5832, GNorm = 7.1026, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.447029
Epoch 110
Validation binary_cross_entropy = 0.454119
Epoch 111
Validation binary_cross_entropy = 0.487162
Epoch 112
Validation binary_cross_entropy = 0.514666
Epoch 113
Validation binary_cross_entropy = 0.549762
Epoch 114
Loss = 5.7091e-01, PNorm = 27.5882, GNorm = 8.4257, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.587724
Epoch 115
Validation binary_cross_entropy = 0.605040
Epoch 116
Validation binary_cross_entropy = 0.602364
Epoch 117
Validation binary_cross_entropy = 0.589727
Epoch 118
Validation binary_cross_entropy = 0.567167
Epoch 119
Loss = 6.2704e-01, PNorm = 27.5930, GNorm = 8.0354, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.541878
Epoch 120
Validation binary_cross_entropy = 0.516746
Epoch 121
Validation binary_cross_entropy = 0.500044
Epoch 122
Validation binary_cross_entropy = 0.493198
Epoch 123
Validation binary_cross_entropy = 0.501711
Epoch 124
Loss = 6.4488e-01, PNorm = 27.5984, GNorm = 9.1727, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.516735
Epoch 125
Validation binary_cross_entropy = 0.544039
Epoch 126
Validation binary_cross_entropy = 0.575980
Epoch 127
Validation binary_cross_entropy = 0.589396
Epoch 128
Validation binary_cross_entropy = 0.588448
Epoch 129
Loss = 4.6169e-01, PNorm = 27.6032, GNorm = 7.5056, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.598598
Epoch 130
Validation binary_cross_entropy = 0.606937
Epoch 131
Validation binary_cross_entropy = 0.612987
Epoch 132
Validation binary_cross_entropy = 0.619266
Epoch 133
Validation binary_cross_entropy = 0.628678
Epoch 134
Loss = 6.4995e-01, PNorm = 27.6079, GNorm = 6.4753, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.638141
Epoch 135
Validation binary_cross_entropy = 0.647281
Epoch 136
Validation binary_cross_entropy = 0.640686
Epoch 137
Validation binary_cross_entropy = 0.628758
Epoch 138
Validation binary_cross_entropy = 0.632409
Epoch 139
Loss = 5.6030e-01, PNorm = 27.6130, GNorm = 5.4775, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.633125
Epoch 140
Validation binary_cross_entropy = 0.652271
Epoch 141
Validation binary_cross_entropy = 0.658437
Epoch 142
Validation binary_cross_entropy = 0.648051
Epoch 143
Validation binary_cross_entropy = 0.621740
Epoch 144
Loss = 5.1342e-01, PNorm = 27.6181, GNorm = 6.9414, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.602305
Epoch 145
Validation binary_cross_entropy = 0.608128
Epoch 146
Validation binary_cross_entropy = 0.605753
Epoch 147
Validation binary_cross_entropy = 0.573850
Epoch 148
Validation binary_cross_entropy = 0.549662
Epoch 149
Loss = 4.6970e-01, PNorm = 27.6218, GNorm = 5.5483, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.534629
Epoch 150
Validation binary_cross_entropy = 0.527282
Epoch 151
Validation binary_cross_entropy = 0.519472
Epoch 152
Validation binary_cross_entropy = 0.518406
Epoch 153
Validation binary_cross_entropy = 0.520079
Epoch 154
Loss = 6.1913e-01, PNorm = 27.6256, GNorm = 5.3992, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.522401
Epoch 155
Validation binary_cross_entropy = 0.526689
Epoch 156
Validation binary_cross_entropy = 0.534179
Epoch 157
Validation binary_cross_entropy = 0.538609
Epoch 158
Validation binary_cross_entropy = 0.531656
Epoch 159
Loss = 6.1406e-01, PNorm = 27.6299, GNorm = 8.3035, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.526545
Epoch 160
Validation binary_cross_entropy = 0.527225
Epoch 161
Validation binary_cross_entropy = 0.524758
Epoch 162
Validation binary_cross_entropy = 0.531342
Epoch 163
Validation binary_cross_entropy = 0.546973
Epoch 164
Loss = 4.2328e-01, PNorm = 27.6345, GNorm = 5.3354, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.564922
Epoch 165
Validation binary_cross_entropy = 0.584864
Epoch 166
Validation binary_cross_entropy = 0.602581
Epoch 167
Validation binary_cross_entropy = 0.594774
Epoch 168
Validation binary_cross_entropy = 0.568720
Epoch 169
Loss = 5.4619e-01, PNorm = 27.6385, GNorm = 7.4801, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.554076
Epoch 170
Validation binary_cross_entropy = 0.538511
Epoch 171
Validation binary_cross_entropy = 0.544033
Epoch 172
Validation binary_cross_entropy = 0.557717
Epoch 173
Validation binary_cross_entropy = 0.577898
Epoch 174
Loss = 6.9415e-01, PNorm = 27.6417, GNorm = 9.0287, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.592152
Epoch 175
Validation binary_cross_entropy = 0.603104
Epoch 176
Validation binary_cross_entropy = 0.606084
Epoch 177
Validation binary_cross_entropy = 0.612285
Epoch 178
Validation binary_cross_entropy = 0.621106
Epoch 179
Loss = 5.3837e-01, PNorm = 27.6457, GNorm = 7.7561, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.633702
Epoch 180
Validation binary_cross_entropy = 0.648681
Epoch 181
Validation binary_cross_entropy = 0.658271
Epoch 182
Validation binary_cross_entropy = 0.654544
Epoch 183
Validation binary_cross_entropy = 0.648282
Epoch 184
Loss = 3.9975e-01, PNorm = 27.6500, GNorm = 4.4309, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.647243
Epoch 185
Validation binary_cross_entropy = 0.635243
Epoch 186
Validation binary_cross_entropy = 0.630399
Epoch 187
Validation binary_cross_entropy = 0.632420
Epoch 188
Validation binary_cross_entropy = 0.628609
Epoch 189
Loss = 5.6116e-01, PNorm = 27.6539, GNorm = 5.0008, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.641160
Epoch 190
Validation binary_cross_entropy = 0.645713
Epoch 191
Validation binary_cross_entropy = 0.642550
Epoch 192
Validation binary_cross_entropy = 0.633526
Epoch 193
Validation binary_cross_entropy = 0.622423
Epoch 194
Loss = 5.2483e-01, PNorm = 27.6578, GNorm = 5.5609, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.612742
Epoch 195
Validation binary_cross_entropy = 0.611108
Epoch 196
Validation binary_cross_entropy = 0.607276
Epoch 197
Validation binary_cross_entropy = 0.602321
Epoch 198
Validation binary_cross_entropy = 0.602960
Epoch 199
Loss = 5.3304e-01, PNorm = 27.6625, GNorm = 5.8769, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.611062
Epoch 200
Validation binary_cross_entropy = 0.620133
Epoch 201
Validation binary_cross_entropy = 0.620398
Epoch 202
Validation binary_cross_entropy = 0.607379
Epoch 203
Validation binary_cross_entropy = 0.597863
Epoch 204
Loss = 5.2619e-01, PNorm = 27.6675, GNorm = 6.0559, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.587004
Epoch 205
Validation binary_cross_entropy = 0.584918
Epoch 206
Validation binary_cross_entropy = 0.589981
Epoch 207
Validation binary_cross_entropy = 0.593454
Epoch 208
Validation binary_cross_entropy = 0.599331
Epoch 209
Loss = 4.6972e-01, PNorm = 27.6721, GNorm = 6.0651, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.599582
Epoch 210
Validation binary_cross_entropy = 0.595875
Epoch 211
Validation binary_cross_entropy = 0.592792
Epoch 212
Validation binary_cross_entropy = 0.587707
Epoch 213
Validation binary_cross_entropy = 0.581021
Epoch 214
Loss = 4.3604e-01, PNorm = 27.6766, GNorm = 3.7157, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.582170
Epoch 215
Validation binary_cross_entropy = 0.587830
Epoch 216
Validation binary_cross_entropy = 0.598826
Epoch 217
Validation binary_cross_entropy = 0.594407
Epoch 218
Validation binary_cross_entropy = 0.582897
Epoch 219
Loss = 6.6245e-01, PNorm = 27.6801, GNorm = 6.0023, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.578339
Epoch 220
Validation binary_cross_entropy = 0.577168
Epoch 221
Validation binary_cross_entropy = 0.583480
Epoch 222
Validation binary_cross_entropy = 0.586890
Epoch 223
Validation binary_cross_entropy = 0.585581
Epoch 224
Loss = 5.1155e-01, PNorm = 27.6844, GNorm = 6.6388, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.576892
Epoch 225
Validation binary_cross_entropy = 0.564300
Epoch 226
Validation binary_cross_entropy = 0.560594
Epoch 227
Validation binary_cross_entropy = 0.559473
Epoch 228
Validation binary_cross_entropy = 0.563962
Epoch 229
Loss = 4.8167e-01, PNorm = 27.6889, GNorm = 5.0397, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.578371
Epoch 230
Validation binary_cross_entropy = 0.593060
Epoch 231
Validation binary_cross_entropy = 0.598127
Epoch 232
Validation binary_cross_entropy = 0.596508
Epoch 233
Validation binary_cross_entropy = 0.594140
Epoch 234
Loss = 5.5890e-01, PNorm = 27.6930, GNorm = 6.2596, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.589010
Epoch 235
Validation binary_cross_entropy = 0.579335
Epoch 236
Validation binary_cross_entropy = 0.573630
Epoch 237
Validation binary_cross_entropy = 0.574143
Epoch 238
Validation binary_cross_entropy = 0.577496
Epoch 239
Loss = 3.8861e-01, PNorm = 27.6973, GNorm = 3.4407, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.581895
Epoch 240
Validation binary_cross_entropy = 0.588975
Epoch 241
Validation binary_cross_entropy = 0.594201
Epoch 242
Validation binary_cross_entropy = 0.599289
Epoch 243
Validation binary_cross_entropy = 0.604279
Epoch 244
Loss = 3.7179e-01, PNorm = 27.7018, GNorm = 3.8229, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.612459
Epoch 245
Validation binary_cross_entropy = 0.621564
Epoch 246
Validation binary_cross_entropy = 0.622120
Epoch 247
Validation binary_cross_entropy = 0.624272
Epoch 248
Validation binary_cross_entropy = 0.625316
Epoch 249
Loss = 4.6214e-01, PNorm = 27.7060, GNorm = 6.2806, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.627132
Epoch 250
Validation binary_cross_entropy = 0.622935
Epoch 251
Validation binary_cross_entropy = 0.612279
Epoch 252
Validation binary_cross_entropy = 0.603814
Epoch 253
Validation binary_cross_entropy = 0.599663
Epoch 254
Loss = 5.2525e-01, PNorm = 27.7093, GNorm = 6.7368, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.595454
Epoch 255
Validation binary_cross_entropy = 0.585189
Epoch 256
Validation binary_cross_entropy = 0.572390
Epoch 257
Validation binary_cross_entropy = 0.562774
Epoch 258
Validation binary_cross_entropy = 0.549994
Epoch 259
Loss = 4.8588e-01, PNorm = 27.7126, GNorm = 5.1492, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.537272
Epoch 260
Validation binary_cross_entropy = 0.528353
Epoch 261
Validation binary_cross_entropy = 0.527133
Epoch 262
Validation binary_cross_entropy = 0.533199
Epoch 263
Validation binary_cross_entropy = 0.535529
Epoch 264
Loss = 4.1899e-01, PNorm = 27.7163, GNorm = 3.7910, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.536991
Epoch 265
Validation binary_cross_entropy = 0.543639
Epoch 266
Validation binary_cross_entropy = 0.553987
Epoch 267
Validation binary_cross_entropy = 0.561704
Epoch 268
Validation binary_cross_entropy = 0.567971
Epoch 269
Loss = 3.7514e-01, PNorm = 27.7198, GNorm = 3.3403, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.576480
Epoch 270
Validation binary_cross_entropy = 0.584684
Epoch 271
Validation binary_cross_entropy = 0.586189
Epoch 272
Validation binary_cross_entropy = 0.580716
Epoch 273
Validation binary_cross_entropy = 0.576034
Epoch 274
Loss = 4.6355e-01, PNorm = 27.7237, GNorm = 5.2890, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.570876
Epoch 275
Validation binary_cross_entropy = 0.570121
Epoch 276
Validation binary_cross_entropy = 0.571864
Epoch 277
Validation binary_cross_entropy = 0.570617
Epoch 278
Validation binary_cross_entropy = 0.568559
Epoch 279
Loss = 5.2868e-01, PNorm = 27.7279, GNorm = 5.5820, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.570037
Epoch 280
Validation binary_cross_entropy = 0.571025
Epoch 281
Validation binary_cross_entropy = 0.573118
Epoch 282
Validation binary_cross_entropy = 0.582680
Epoch 283
Validation binary_cross_entropy = 0.596300
Epoch 284
Loss = 4.8619e-01, PNorm = 27.7321, GNorm = 5.4859, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.609416
Epoch 285
Validation binary_cross_entropy = 0.622339
Epoch 286
Validation binary_cross_entropy = 0.637673
Epoch 287
Validation binary_cross_entropy = 0.647141
Epoch 288
Validation binary_cross_entropy = 0.647388
Epoch 289
Loss = 3.9015e-01, PNorm = 27.7363, GNorm = 3.9953, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.644051
Epoch 290
Validation binary_cross_entropy = 0.641880
Epoch 291
Validation binary_cross_entropy = 0.634155
Epoch 292
Validation binary_cross_entropy = 0.626115
Epoch 293
Validation binary_cross_entropy = 0.615178
Epoch 294
Loss = 4.4040e-01, PNorm = 27.7398, GNorm = 5.2945, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.606534
Epoch 295
Validation binary_cross_entropy = 0.610301
Epoch 296
Validation binary_cross_entropy = 0.621278
Epoch 297
Validation binary_cross_entropy = 0.638355
Epoch 298
Validation binary_cross_entropy = 0.648290
Epoch 299
Loss = 3.9013e-01, PNorm = 27.7434, GNorm = 3.6257, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.653891
Model 0 best validation binary_cross_entropy = 0.328180 on epoch 21
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.194977
Ensemble test binary_cross_entropy = 0.194977
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 281,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 2.579373
Epoch 1
Validation binary_cross_entropy = 3.658446
Epoch 2
Validation binary_cross_entropy = 0.887712
Epoch 3
Validation binary_cross_entropy = 0.409398
Epoch 4
Loss = 3.9330e+00, PNorm = 27.3561, GNorm = 43.8539, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.408126
Epoch 5
Validation binary_cross_entropy = 0.467972
Epoch 6
Validation binary_cross_entropy = 1.207973
Epoch 7
Validation binary_cross_entropy = 1.535205
Epoch 8
Validation binary_cross_entropy = 0.696394
Epoch 9
Loss = 3.4881e+00, PNorm = 27.3747, GNorm = 47.4257, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.458145
Epoch 10
Validation binary_cross_entropy = 0.521778
Epoch 11
Validation binary_cross_entropy = 0.742092
Epoch 12
Validation binary_cross_entropy = 1.110991
Epoch 13
Validation binary_cross_entropy = 1.283620
Epoch 14
Loss = 1.2350e+00, PNorm = 27.4021, GNorm = 15.3942, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 1.045981
Epoch 15
Validation binary_cross_entropy = 0.482910
Epoch 16
Validation binary_cross_entropy = 0.366250
Epoch 17
Validation binary_cross_entropy = 0.396397
Epoch 18
Validation binary_cross_entropy = 0.350724
Epoch 19
Loss = 1.7132e+00, PNorm = 27.4238, GNorm = 21.4939, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.450883
Epoch 20
Validation binary_cross_entropy = 0.801903
Epoch 21
Validation binary_cross_entropy = 1.209799
Epoch 22
Validation binary_cross_entropy = 0.903078
Epoch 23
Validation binary_cross_entropy = 0.344639
Epoch 24
Loss = 2.0274e+00, PNorm = 27.4406, GNorm = 19.7878, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.319863
Epoch 25
Validation binary_cross_entropy = 0.305157
Epoch 26
Validation binary_cross_entropy = 0.354774
Epoch 27
Validation binary_cross_entropy = 0.704957
Epoch 28
Validation binary_cross_entropy = 0.696097
Epoch 29
Loss = 1.1815e+00, PNorm = 27.4561, GNorm = 19.9753, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.441085
Epoch 30
Validation binary_cross_entropy = 0.307517
Epoch 31
Validation binary_cross_entropy = 0.313895
Epoch 32
Validation binary_cross_entropy = 0.570662
Epoch 33
Validation binary_cross_entropy = 1.128792
Epoch 34
Loss = 1.4537e+00, PNorm = 27.4676, GNorm = 20.1474, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 1.369395
Epoch 35
Validation binary_cross_entropy = 1.209109
Epoch 36
Validation binary_cross_entropy = 1.031312
Epoch 37
Validation binary_cross_entropy = 0.701421
Epoch 38
Validation binary_cross_entropy = 0.878492
Epoch 39
Loss = 1.4557e+00, PNorm = 27.4784, GNorm = 20.9648, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 1.137259
Epoch 40
Validation binary_cross_entropy = 1.437949
Epoch 41
Validation binary_cross_entropy = 1.397986
Epoch 42
Validation binary_cross_entropy = 1.150034
Epoch 43
Validation binary_cross_entropy = 1.036453
Epoch 44
Loss = 1.1581e+00, PNorm = 27.4894, GNorm = 20.0950, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.913532
Epoch 45
Validation binary_cross_entropy = 0.696893
Epoch 46
Validation binary_cross_entropy = 0.581696
Epoch 47
Validation binary_cross_entropy = 0.645455
Epoch 48
Validation binary_cross_entropy = 0.763791
Epoch 49
Loss = 9.3227e-01, PNorm = 27.4998, GNorm = 12.5550, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.775006
Epoch 50
Validation binary_cross_entropy = 0.759088
Epoch 51
Validation binary_cross_entropy = 0.704857
Epoch 52
Validation binary_cross_entropy = 0.584751
Epoch 53
Validation binary_cross_entropy = 0.482106
Epoch 54
Loss = 1.2287e+00, PNorm = 27.5077, GNorm = 11.0421, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.471583
Epoch 55
Validation binary_cross_entropy = 0.481722
Epoch 56
Validation binary_cross_entropy = 0.549355
Epoch 57
Validation binary_cross_entropy = 0.626520
Epoch 58
Validation binary_cross_entropy = 0.626906
Epoch 59
Loss = 8.4375e-01, PNorm = 27.5152, GNorm = 8.9379, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.605036
Epoch 60
Validation binary_cross_entropy = 0.634123
Epoch 61
Validation binary_cross_entropy = 0.727272
Epoch 62
Validation binary_cross_entropy = 0.809523
Epoch 63
Validation binary_cross_entropy = 0.819073
Epoch 64
Loss = 1.0435e+00, PNorm = 27.5226, GNorm = 12.3676, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.692403
Epoch 65
Validation binary_cross_entropy = 0.588820
Epoch 66
Validation binary_cross_entropy = 0.483695
Epoch 67
Validation binary_cross_entropy = 0.448414
Epoch 68
Validation binary_cross_entropy = 0.467324
Epoch 69
Loss = 9.3738e-01, PNorm = 27.5285, GNorm = 13.1269, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.568573
Epoch 70
Validation binary_cross_entropy = 0.686217
Epoch 71
Validation binary_cross_entropy = 0.817327
Epoch 72
Validation binary_cross_entropy = 0.936870
Epoch 73
Validation binary_cross_entropy = 0.796584
Epoch 74
Loss = 7.5221e-01, PNorm = 27.5344, GNorm = 8.5500, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.567505
Epoch 75
Validation binary_cross_entropy = 0.452267
Epoch 76
Validation binary_cross_entropy = 0.424896
Epoch 77
Validation binary_cross_entropy = 0.464909
Epoch 78
Validation binary_cross_entropy = 0.578621
Epoch 79
Loss = 1.1450e+00, PNorm = 27.5410, GNorm = 13.5092, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.734907
Epoch 80
Validation binary_cross_entropy = 0.865463
Epoch 81
Validation binary_cross_entropy = 0.874201
Epoch 82
Validation binary_cross_entropy = 0.825152
Epoch 83
Validation binary_cross_entropy = 0.729001
Epoch 84
Loss = 1.0020e+00, PNorm = 27.5461, GNorm = 16.1363, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.612965
Epoch 85
Validation binary_cross_entropy = 0.487666
Epoch 86
Validation binary_cross_entropy = 0.421938
Epoch 87
Validation binary_cross_entropy = 0.420829
Epoch 88
Validation binary_cross_entropy = 0.466623
Epoch 89
Loss = 7.6998e-01, PNorm = 27.5517, GNorm = 11.1508, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.542165
Epoch 90
Validation binary_cross_entropy = 0.624486
Epoch 91
Validation binary_cross_entropy = 0.696882
Epoch 92
Validation binary_cross_entropy = 0.697056
Epoch 93
Validation binary_cross_entropy = 0.662429
Epoch 94
Loss = 7.9720e-01, PNorm = 27.5580, GNorm = 13.9590, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.615872
Epoch 95
Validation binary_cross_entropy = 0.594072
Epoch 96
Validation binary_cross_entropy = 0.604769
Epoch 97
Validation binary_cross_entropy = 0.616388
Epoch 98
Validation binary_cross_entropy = 0.617315
Epoch 99
Loss = 7.5699e-01, PNorm = 27.5627, GNorm = 12.3257, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.628813
Epoch 100
Validation binary_cross_entropy = 0.640862
Epoch 101
Validation binary_cross_entropy = 0.660188
Epoch 102
Validation binary_cross_entropy = 0.672111
Epoch 103
Validation binary_cross_entropy = 0.681224
Epoch 104
Loss = 6.5154e-01, PNorm = 27.5671, GNorm = 7.1827, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.675289
Epoch 105
Validation binary_cross_entropy = 0.676218
Epoch 106
Validation binary_cross_entropy = 0.663740
Epoch 107
Validation binary_cross_entropy = 0.640485
Epoch 108
Validation binary_cross_entropy = 0.608076
Epoch 109
Loss = 6.4611e-01, PNorm = 27.5714, GNorm = 8.5067, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.577652
Epoch 110
Validation binary_cross_entropy = 0.559653
Epoch 111
Validation binary_cross_entropy = 0.558375
Epoch 112
Validation binary_cross_entropy = 0.580807
Epoch 113
Validation binary_cross_entropy = 0.622032
Epoch 114
Loss = 7.9605e-01, PNorm = 27.5758, GNorm = 13.7091, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.648177
Epoch 115
Validation binary_cross_entropy = 0.664881
Epoch 116
Validation binary_cross_entropy = 0.663875
Epoch 117
Validation binary_cross_entropy = 0.651098
Epoch 118
Validation binary_cross_entropy = 0.632162
Epoch 119
Loss = 6.2347e-01, PNorm = 27.5801, GNorm = 6.8927, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.619707
Epoch 120
Validation binary_cross_entropy = 0.613787
Epoch 121
Validation binary_cross_entropy = 0.592575
Epoch 122
Validation binary_cross_entropy = 0.575087
Epoch 123
Validation binary_cross_entropy = 0.572864
Epoch 124
Loss = 5.9999e-01, PNorm = 27.5851, GNorm = 7.0188, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.568278
Epoch 125
Validation binary_cross_entropy = 0.571666
Epoch 126
Validation binary_cross_entropy = 0.578572
Epoch 127
Validation binary_cross_entropy = 0.585595
Epoch 128
Validation binary_cross_entropy = 0.589033
Epoch 129
Loss = 6.1706e-01, PNorm = 27.5892, GNorm = 8.0375, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.591056
Epoch 130
Validation binary_cross_entropy = 0.597597
Epoch 131
Validation binary_cross_entropy = 0.604136
Epoch 132
Validation binary_cross_entropy = 0.604416
Epoch 133
Validation binary_cross_entropy = 0.602840
Epoch 134
Loss = 5.9240e-01, PNorm = 27.5929, GNorm = 6.4981, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.585859
Epoch 135
Validation binary_cross_entropy = 0.562773
Epoch 136
Validation binary_cross_entropy = 0.561223
Epoch 137
Validation binary_cross_entropy = 0.569526
Epoch 138
Validation binary_cross_entropy = 0.580932
Epoch 139
Loss = 5.4870e-01, PNorm = 27.5965, GNorm = 7.0073, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.599553
Epoch 140
Validation binary_cross_entropy = 0.609508
Epoch 141
Validation binary_cross_entropy = 0.609266
Epoch 142
Validation binary_cross_entropy = 0.616451
Epoch 143
Validation binary_cross_entropy = 0.626059
Epoch 144
Loss = 5.7215e-01, PNorm = 27.6013, GNorm = 9.8806, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.629543
Epoch 145
Validation binary_cross_entropy = 0.646748
Epoch 146
Validation binary_cross_entropy = 0.645870
Epoch 147
Validation binary_cross_entropy = 0.629653
Epoch 148
Validation binary_cross_entropy = 0.614691
Epoch 149
Loss = 7.8660e-01, PNorm = 27.6056, GNorm = 5.4441, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.587117
Epoch 150
Validation binary_cross_entropy = 0.558354
Epoch 151
Validation binary_cross_entropy = 0.537966
Epoch 152
Validation binary_cross_entropy = 0.532139
Epoch 153
Validation binary_cross_entropy = 0.548081
Epoch 154
Loss = 6.4207e-01, PNorm = 27.6090, GNorm = 6.0465, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.569689
Epoch 155
Validation binary_cross_entropy = 0.582815
Epoch 156
Validation binary_cross_entropy = 0.594505
Epoch 157
Validation binary_cross_entropy = 0.598800
Epoch 158
Validation binary_cross_entropy = 0.581190
Epoch 159
Loss = 4.3937e-01, PNorm = 27.6130, GNorm = 5.1337, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.554626
Epoch 160
Validation binary_cross_entropy = 0.527223
Epoch 161
Validation binary_cross_entropy = 0.515808
Epoch 162
Validation binary_cross_entropy = 0.508067
Epoch 163
Validation binary_cross_entropy = 0.498690
Epoch 164
Loss = 5.9278e-01, PNorm = 27.6163, GNorm = 6.4930, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.488014
Epoch 165
Validation binary_cross_entropy = 0.488200
Epoch 166
Validation binary_cross_entropy = 0.493120
Epoch 167
Validation binary_cross_entropy = 0.512803
Epoch 168
Validation binary_cross_entropy = 0.534617
Epoch 169
Loss = 5.9507e-01, PNorm = 27.6198, GNorm = 4.8392, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.539489
Epoch 170
Validation binary_cross_entropy = 0.532921
Epoch 171
Validation binary_cross_entropy = 0.516659
Epoch 172
Validation binary_cross_entropy = 0.502616
Epoch 173
Validation binary_cross_entropy = 0.488122
Epoch 174
Loss = 4.7529e-01, PNorm = 27.6234, GNorm = 4.9081, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.476674
Epoch 175
Validation binary_cross_entropy = 0.476307
Epoch 176
Validation binary_cross_entropy = 0.473552
Epoch 177
Validation binary_cross_entropy = 0.486877
Epoch 178
Validation binary_cross_entropy = 0.491430
Epoch 179
Loss = 6.0457e-01, PNorm = 27.6270, GNorm = 7.6662, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.490089
Epoch 180
Validation binary_cross_entropy = 0.500335
Epoch 181
Validation binary_cross_entropy = 0.511486
Epoch 182
Validation binary_cross_entropy = 0.512342
Epoch 183
Validation binary_cross_entropy = 0.510693
Epoch 184
Loss = 4.5648e-01, PNorm = 27.6302, GNorm = 5.6035, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.506581
Epoch 185
Validation binary_cross_entropy = 0.502402
Epoch 186
Validation binary_cross_entropy = 0.506924
Epoch 187
Validation binary_cross_entropy = 0.510886
Epoch 188
Validation binary_cross_entropy = 0.514122
Epoch 189
Loss = 5.0910e-01, PNorm = 27.6336, GNorm = 6.0100, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.519672
Epoch 190
Validation binary_cross_entropy = 0.524358
Epoch 191
Validation binary_cross_entropy = 0.533779
Epoch 192
Validation binary_cross_entropy = 0.546180
Epoch 193
Validation binary_cross_entropy = 0.556207
Epoch 194
Loss = 5.9742e-01, PNorm = 27.6372, GNorm = 5.4616, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.568933
Epoch 195
Validation binary_cross_entropy = 0.565414
Epoch 196
Validation binary_cross_entropy = 0.548174
Epoch 197
Validation binary_cross_entropy = 0.528714
Epoch 198
Validation binary_cross_entropy = 0.510639
Epoch 199
Loss = 6.7995e-01, PNorm = 27.6410, GNorm = 6.3260, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.496872
Epoch 200
Validation binary_cross_entropy = 0.497674
Epoch 201
Validation binary_cross_entropy = 0.504357
Epoch 202
Validation binary_cross_entropy = 0.507568
Epoch 203
Validation binary_cross_entropy = 0.511822
Epoch 204
Loss = 4.4870e-01, PNorm = 27.6451, GNorm = 4.6733, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.513393
Epoch 205
Validation binary_cross_entropy = 0.514007
Epoch 206
Validation binary_cross_entropy = 0.523004
Epoch 207
Validation binary_cross_entropy = 0.522527
Epoch 208
Validation binary_cross_entropy = 0.530697
Epoch 209
Loss = 5.1900e-01, PNorm = 27.6491, GNorm = 5.8546, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.531545
Epoch 210
Validation binary_cross_entropy = 0.536808
Epoch 211
Validation binary_cross_entropy = 0.540556
Epoch 212
Validation binary_cross_entropy = 0.540203
Epoch 213
Validation binary_cross_entropy = 0.537634
Epoch 214
Loss = 4.3306e-01, PNorm = 27.6525, GNorm = 4.8498, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.536391
Epoch 215
Validation binary_cross_entropy = 0.538587
Epoch 216
Validation binary_cross_entropy = 0.538668
Epoch 217
Validation binary_cross_entropy = 0.538204
Epoch 218
Validation binary_cross_entropy = 0.539707
Epoch 219
Loss = 5.1268e-01, PNorm = 27.6560, GNorm = 4.2128, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.538425
Epoch 220
Validation binary_cross_entropy = 0.535184
Epoch 221
Validation binary_cross_entropy = 0.535672
Epoch 222
Validation binary_cross_entropy = 0.536853
Epoch 223
Validation binary_cross_entropy = 0.543502
Epoch 224
Loss = 4.5685e-01, PNorm = 27.6605, GNorm = 3.9969, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.546870
Epoch 225
Validation binary_cross_entropy = 0.552473
Epoch 226
Validation binary_cross_entropy = 0.558563
Epoch 227
Validation binary_cross_entropy = 0.562968
Epoch 228
Validation binary_cross_entropy = 0.566718
Epoch 229
Loss = 5.3548e-01, PNorm = 27.6652, GNorm = 5.2981, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.567090
Epoch 230
Validation binary_cross_entropy = 0.564073
Epoch 231
Validation binary_cross_entropy = 0.566271
Epoch 232
Validation binary_cross_entropy = 0.571955
Epoch 233
Validation binary_cross_entropy = 0.586730
Epoch 234
Loss = 4.9701e-01, PNorm = 27.6695, GNorm = 4.9973, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.600336
Epoch 235
Validation binary_cross_entropy = 0.607001
Epoch 236
Validation binary_cross_entropy = 0.613436
Epoch 237
Validation binary_cross_entropy = 0.620710
Epoch 238
Validation binary_cross_entropy = 0.630601
Epoch 239
Loss = 5.0293e-01, PNorm = 27.6744, GNorm = 5.0144, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.637203
Epoch 240
Validation binary_cross_entropy = 0.632918
Epoch 241
Validation binary_cross_entropy = 0.625757
Epoch 242
Validation binary_cross_entropy = 0.619576
Epoch 243
Validation binary_cross_entropy = 0.612186
Epoch 244
Loss = 4.4338e-01, PNorm = 27.6787, GNorm = 4.0729, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.603945
Epoch 245
Validation binary_cross_entropy = 0.593126
Epoch 246
Validation binary_cross_entropy = 0.583280
Epoch 247
Validation binary_cross_entropy = 0.583194
Epoch 248
Validation binary_cross_entropy = 0.584201
Epoch 249
Loss = 5.7087e-01, PNorm = 27.6829, GNorm = 4.6066, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.592491
Epoch 250
Validation binary_cross_entropy = 0.605522
Epoch 251
Validation binary_cross_entropy = 0.617690
Epoch 252
Validation binary_cross_entropy = 0.631209
Epoch 253
Validation binary_cross_entropy = 0.632955
Epoch 254
Loss = 5.2093e-01, PNorm = 27.6865, GNorm = 4.5984, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.618930
Epoch 255
Validation binary_cross_entropy = 0.601896
Epoch 256
Validation binary_cross_entropy = 0.582821
Epoch 257
Validation binary_cross_entropy = 0.571638
Epoch 258
Validation binary_cross_entropy = 0.569113
Epoch 259
Loss = 4.8517e-01, PNorm = 27.6907, GNorm = 5.5912, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.569108
Epoch 260
Validation binary_cross_entropy = 0.566046
Epoch 261
Validation binary_cross_entropy = 0.566444
Epoch 262
Validation binary_cross_entropy = 0.568129
Epoch 263
Validation binary_cross_entropy = 0.572106
Epoch 264
Loss = 4.9606e-01, PNorm = 27.6943, GNorm = 5.7744, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.577624
Epoch 265
Validation binary_cross_entropy = 0.584438
Epoch 266
Validation binary_cross_entropy = 0.588824
Epoch 267
Validation binary_cross_entropy = 0.590854
Epoch 268
Validation binary_cross_entropy = 0.592389
Epoch 269
Loss = 4.7078e-01, PNorm = 27.6975, GNorm = 3.7507, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.598147
Epoch 270
Validation binary_cross_entropy = 0.601185
Epoch 271
Validation binary_cross_entropy = 0.595400
Epoch 272
Validation binary_cross_entropy = 0.593865
Epoch 273
Validation binary_cross_entropy = 0.600107
Epoch 274
Loss = 4.5618e-01, PNorm = 27.7015, GNorm = 4.3269, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.601983
Epoch 275
Validation binary_cross_entropy = 0.600923
Epoch 276
Validation binary_cross_entropy = 0.600287
Epoch 277
Validation binary_cross_entropy = 0.589897
Epoch 278
Validation binary_cross_entropy = 0.574516
Epoch 279
Loss = 4.9706e-01, PNorm = 27.7053, GNorm = 4.8919, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.564470
Epoch 280
Validation binary_cross_entropy = 0.559065
Epoch 281
Validation binary_cross_entropy = 0.555655
Epoch 282
Validation binary_cross_entropy = 0.553414
Epoch 283
Validation binary_cross_entropy = 0.549634
Epoch 284
Loss = 5.1038e-01, PNorm = 27.7086, GNorm = 4.3510, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.551383
Epoch 285
Validation binary_cross_entropy = 0.546615
Epoch 286
Validation binary_cross_entropy = 0.543073
Epoch 287
Validation binary_cross_entropy = 0.541985
Epoch 288
Validation binary_cross_entropy = 0.542784
Epoch 289
Loss = 4.4217e-01, PNorm = 27.7123, GNorm = 4.1953, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.549559
Epoch 290
Validation binary_cross_entropy = 0.555906
Epoch 291
Validation binary_cross_entropy = 0.556338
Epoch 292
Validation binary_cross_entropy = 0.557055
Epoch 293
Validation binary_cross_entropy = 0.559681
Epoch 294
Loss = 3.9954e-01, PNorm = 27.7153, GNorm = 5.3437, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.564641
Epoch 295
Validation binary_cross_entropy = 0.569851
Epoch 296
Validation binary_cross_entropy = 0.579448
Epoch 297
Validation binary_cross_entropy = 0.586621
Epoch 298
Validation binary_cross_entropy = 0.588788
Epoch 299
Loss = 6.0849e-01, PNorm = 27.7186, GNorm = 5.8916, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.589939
Model 0 best validation binary_cross_entropy = 0.305157 on epoch 25
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.223320
Ensemble test binary_cross_entropy = 0.223320
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 281,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 1.381727
Epoch 1
Validation binary_cross_entropy = 2.400867
Epoch 2
Validation binary_cross_entropy = 0.726219
Epoch 3
Validation binary_cross_entropy = 0.369693
Epoch 4
Loss = 3.4927e+00, PNorm = 27.3556, GNorm = 37.6379, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.389817
Epoch 5
Validation binary_cross_entropy = 0.430057
Epoch 6
Validation binary_cross_entropy = 0.734913
Epoch 7
Validation binary_cross_entropy = 0.881560
Epoch 8
Validation binary_cross_entropy = 0.917205
Epoch 9
Loss = 2.9838e+00, PNorm = 27.3714, GNorm = 36.3328, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.793689
Epoch 10
Validation binary_cross_entropy = 0.528679
Epoch 11
Validation binary_cross_entropy = 0.622301
Epoch 12
Validation binary_cross_entropy = 0.934369
Epoch 13
Validation binary_cross_entropy = 0.802203
Epoch 14
Loss = 1.8474e+00, PNorm = 27.3977, GNorm = 30.9679, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.720963
Epoch 15
Validation binary_cross_entropy = 1.270185
Epoch 16
Validation binary_cross_entropy = 2.568156
Epoch 17
Validation binary_cross_entropy = 1.760272
Epoch 18
Validation binary_cross_entropy = 0.878787
Epoch 19
Loss = 2.0866e+00, PNorm = 27.4236, GNorm = 22.5542, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.633856
Epoch 20
Validation binary_cross_entropy = 0.708280
Epoch 21
Validation binary_cross_entropy = 1.266422
Epoch 22
Validation binary_cross_entropy = 1.877529
Epoch 23
Validation binary_cross_entropy = 1.120073
Epoch 24
Loss = 1.5885e+00, PNorm = 27.4440, GNorm = 23.2456, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.538574
Epoch 25
Validation binary_cross_entropy = 0.515394
Epoch 26
Validation binary_cross_entropy = 0.821336
Epoch 27
Validation binary_cross_entropy = 1.221477
Epoch 28
Validation binary_cross_entropy = 1.505206
Epoch 29
Loss = 1.6751e+00, PNorm = 27.4617, GNorm = 28.8579, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 1.092696
Epoch 30
Validation binary_cross_entropy = 0.561810
Epoch 31
Validation binary_cross_entropy = 0.400989
Epoch 32
Validation binary_cross_entropy = 0.404811
Epoch 33
Validation binary_cross_entropy = 0.486004
Epoch 34
Loss = 1.2718e+00, PNorm = 27.4742, GNorm = 15.6782, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.719039
Epoch 35
Validation binary_cross_entropy = 0.782429
Epoch 36
Validation binary_cross_entropy = 0.661342
Epoch 37
Validation binary_cross_entropy = 0.628807
Epoch 38
Validation binary_cross_entropy = 0.562569
Epoch 39
Loss = 1.1930e+00, PNorm = 27.4852, GNorm = 23.1210, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.569597
Epoch 40
Validation binary_cross_entropy = 0.678759
Epoch 41
Validation binary_cross_entropy = 0.694427
Epoch 42
Validation binary_cross_entropy = 0.814184
Epoch 43
Validation binary_cross_entropy = 0.861205
Epoch 44
Loss = 1.1773e+00, PNorm = 27.4951, GNorm = 16.4202, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.858586
Epoch 45
Validation binary_cross_entropy = 0.770009
Epoch 46
Validation binary_cross_entropy = 0.737246
Epoch 47
Validation binary_cross_entropy = 0.762520
Epoch 48
Validation binary_cross_entropy = 0.780836
Epoch 49
Loss = 1.0136e+00, PNorm = 27.5046, GNorm = 13.2530, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.760623
Epoch 50
Validation binary_cross_entropy = 0.826418
Epoch 51
Validation binary_cross_entropy = 0.835577
Epoch 52
Validation binary_cross_entropy = 0.744513
Epoch 53
Validation binary_cross_entropy = 0.691236
Epoch 54
Loss = 6.9066e-01, PNorm = 27.5134, GNorm = 11.2020, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.616793
Epoch 55
Validation binary_cross_entropy = 0.609057
Epoch 56
Validation binary_cross_entropy = 0.751036
Epoch 57
Validation binary_cross_entropy = 0.844425
Epoch 58
Validation binary_cross_entropy = 0.854739
Epoch 59
Loss = 5.5344e-01, PNorm = 27.5210, GNorm = 9.6664, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.849014
Epoch 60
Validation binary_cross_entropy = 0.720976
Epoch 61
Validation binary_cross_entropy = 0.675131
Epoch 62
Validation binary_cross_entropy = 0.622130
Epoch 63
Validation binary_cross_entropy = 0.597928
Epoch 64
Loss = 8.9192e-01, PNorm = 27.5295, GNorm = 14.3334, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.590009
Epoch 65
Validation binary_cross_entropy = 0.600469
Epoch 66
Validation binary_cross_entropy = 0.566553
Epoch 67
Validation binary_cross_entropy = 0.548058
Epoch 68
Validation binary_cross_entropy = 0.513560
Epoch 69
Loss = 9.9950e-01, PNorm = 27.5363, GNorm = 10.7848, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.487259
Epoch 70
Validation binary_cross_entropy = 0.471583
Epoch 71
Validation binary_cross_entropy = 0.473318
Epoch 72
Validation binary_cross_entropy = 0.477293
Epoch 73
Validation binary_cross_entropy = 0.464669
Epoch 74
Loss = 6.2596e-01, PNorm = 27.5410, GNorm = 12.3123, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.449850
Epoch 75
Validation binary_cross_entropy = 0.446664
Epoch 76
Validation binary_cross_entropy = 0.444148
Epoch 77
Validation binary_cross_entropy = 0.451889
Epoch 78
Validation binary_cross_entropy = 0.470194
Epoch 79
Loss = 8.7604e-01, PNorm = 27.5461, GNorm = 14.3493, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.479183
Epoch 80
Validation binary_cross_entropy = 0.483062
Epoch 81
Validation binary_cross_entropy = 0.491374
Epoch 82
Validation binary_cross_entropy = 0.508511
Epoch 83
Validation binary_cross_entropy = 0.538839
Epoch 84
Loss = 7.3832e-01, PNorm = 27.5517, GNorm = 13.1957, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.556097
Epoch 85
Validation binary_cross_entropy = 0.545574
Epoch 86
Validation binary_cross_entropy = 0.522889
Epoch 87
Validation binary_cross_entropy = 0.513507
Epoch 88
Validation binary_cross_entropy = 0.514357
Epoch 89
Loss = 8.8764e-01, PNorm = 27.5567, GNorm = 11.3238, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.511882
Epoch 90
Validation binary_cross_entropy = 0.524319
Epoch 91
Validation binary_cross_entropy = 0.534979
Epoch 92
Validation binary_cross_entropy = 0.565033
Epoch 93
Validation binary_cross_entropy = 0.623773
Epoch 94
Loss = 5.7760e-01, PNorm = 27.5625, GNorm = 8.5588, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.681650
Epoch 95
Validation binary_cross_entropy = 0.702545
Epoch 96
Validation binary_cross_entropy = 0.680044
Epoch 97
Validation binary_cross_entropy = 0.628168
Epoch 98
Validation binary_cross_entropy = 0.585703
Epoch 99
Loss = 7.2887e-01, PNorm = 27.5686, GNorm = 7.0445, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.590680
Epoch 100
Validation binary_cross_entropy = 0.580312
Epoch 101
Validation binary_cross_entropy = 0.582555
Epoch 102
Validation binary_cross_entropy = 0.581543
Epoch 103
Validation binary_cross_entropy = 0.592183
Epoch 104
Loss = 6.3527e-01, PNorm = 27.5743, GNorm = 9.6709, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.608104
Epoch 105
Validation binary_cross_entropy = 0.605282
Epoch 106
Validation binary_cross_entropy = 0.611792
Epoch 107
Validation binary_cross_entropy = 0.606340
Epoch 108
Validation binary_cross_entropy = 0.612835
Epoch 109
Loss = 6.6847e-01, PNorm = 27.5794, GNorm = 6.3484, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.605917
Epoch 110
Validation binary_cross_entropy = 0.582341
Epoch 111
Validation binary_cross_entropy = 0.572114
Epoch 112
Validation binary_cross_entropy = 0.574883
Epoch 113
Validation binary_cross_entropy = 0.584353
Epoch 114
Loss = 6.7217e-01, PNorm = 27.5846, GNorm = 9.4137, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.601738
Epoch 115
Validation binary_cross_entropy = 0.634760
Epoch 116
Validation binary_cross_entropy = 0.670851
Epoch 117
Validation binary_cross_entropy = 0.692886
Epoch 118
Validation binary_cross_entropy = 0.707313
Epoch 119
Loss = 8.2147e-01, PNorm = 27.5896, GNorm = 9.4739, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.689493
Epoch 120
Validation binary_cross_entropy = 0.658633
Epoch 121
Validation binary_cross_entropy = 0.638157
Epoch 122
Validation binary_cross_entropy = 0.618851
Epoch 123
Validation binary_cross_entropy = 0.610581
Epoch 124
Loss = 8.0700e-01, PNorm = 27.5939, GNorm = 9.8595, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.600012
Epoch 125
Validation binary_cross_entropy = 0.606973
Epoch 126
Validation binary_cross_entropy = 0.607100
Epoch 127
Validation binary_cross_entropy = 0.614272
Epoch 128
Validation binary_cross_entropy = 0.621787
Epoch 129
Loss = 5.6777e-01, PNorm = 27.5981, GNorm = 8.8339, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.643368
Epoch 130
Validation binary_cross_entropy = 0.662488
Epoch 131
Validation binary_cross_entropy = 0.668957
Epoch 132
Validation binary_cross_entropy = 0.666799
Epoch 133
Validation binary_cross_entropy = 0.650337
Epoch 134
Loss = 5.1808e-01, PNorm = 27.6029, GNorm = 6.6575, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.632270
Epoch 135
Validation binary_cross_entropy = 0.615918
Epoch 136
Validation binary_cross_entropy = 0.596114
Epoch 137
Validation binary_cross_entropy = 0.564742
Epoch 138
Validation binary_cross_entropy = 0.539460
Epoch 139
Loss = 5.7934e-01, PNorm = 27.6065, GNorm = 7.0106, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.526905
Epoch 140
Validation binary_cross_entropy = 0.509458
Epoch 141
Validation binary_cross_entropy = 0.497117
Epoch 142
Validation binary_cross_entropy = 0.490588
Epoch 143
Validation binary_cross_entropy = 0.500156
Epoch 144
Loss = 5.2794e-01, PNorm = 27.6108, GNorm = 8.7281, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.524803
Epoch 145
Validation binary_cross_entropy = 0.546585
Epoch 146
Validation binary_cross_entropy = 0.554853
Epoch 147
Validation binary_cross_entropy = 0.546893
Epoch 148
Validation binary_cross_entropy = 0.529260
Epoch 149
Loss = 5.9895e-01, PNorm = 27.6159, GNorm = 8.1182, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.505724
Epoch 150
Validation binary_cross_entropy = 0.487208
Epoch 151
Validation binary_cross_entropy = 0.473660
Epoch 152
Validation binary_cross_entropy = 0.465318
Epoch 153
Validation binary_cross_entropy = 0.465276
Epoch 154
Loss = 4.6617e-01, PNorm = 27.6205, GNorm = 6.1068, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.476205
Epoch 155
Validation binary_cross_entropy = 0.487687
Epoch 156
Validation binary_cross_entropy = 0.506310
Epoch 157
Validation binary_cross_entropy = 0.527251
Epoch 158
Validation binary_cross_entropy = 0.547348
Epoch 159
Loss = 5.5827e-01, PNorm = 27.6253, GNorm = 4.3776, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.571954
Epoch 160
Validation binary_cross_entropy = 0.593008
Epoch 161
Validation binary_cross_entropy = 0.609086
Epoch 162
Validation binary_cross_entropy = 0.611702
Epoch 163
Validation binary_cross_entropy = 0.581921
Epoch 164
Loss = 6.4337e-01, PNorm = 27.6297, GNorm = 7.1328, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.547860
Epoch 165
Validation binary_cross_entropy = 0.536883
Epoch 166
Validation binary_cross_entropy = 0.535300
Epoch 167
Validation binary_cross_entropy = 0.542265
Epoch 168
Validation binary_cross_entropy = 0.553635
Epoch 169
Loss = 5.2549e-01, PNorm = 27.6328, GNorm = 5.2615, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.565061
Epoch 170
Validation binary_cross_entropy = 0.576000
Epoch 171
Validation binary_cross_entropy = 0.585178
Epoch 172
Validation binary_cross_entropy = 0.575270
Epoch 173
Validation binary_cross_entropy = 0.577722
Epoch 174
Loss = 5.3380e-01, PNorm = 27.6362, GNorm = 5.5145, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.576530
Epoch 175
Validation binary_cross_entropy = 0.561124
Epoch 176
Validation binary_cross_entropy = 0.544007
Epoch 177
Validation binary_cross_entropy = 0.545656
Epoch 178
Validation binary_cross_entropy = 0.548576
Epoch 179
Loss = 5.0400e-01, PNorm = 27.6404, GNorm = 7.6095, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.554431
Epoch 180
Validation binary_cross_entropy = 0.563592
Epoch 181
Validation binary_cross_entropy = 0.547066
Epoch 182
Validation binary_cross_entropy = 0.513818
Epoch 183
Validation binary_cross_entropy = 0.493168
Epoch 184
Loss = 4.9884e-01, PNorm = 27.6450, GNorm = 5.2263, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.500814
Epoch 185
Validation binary_cross_entropy = 0.519191
Epoch 186
Validation binary_cross_entropy = 0.540085
Epoch 187
Validation binary_cross_entropy = 0.557022
Epoch 188
Validation binary_cross_entropy = 0.571694
Epoch 189
Loss = 5.1805e-01, PNorm = 27.6495, GNorm = 5.9353, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.578927
Epoch 190
Validation binary_cross_entropy = 0.589361
Epoch 191
Validation binary_cross_entropy = 0.596560
Epoch 192
Validation binary_cross_entropy = 0.595478
Epoch 193
Validation binary_cross_entropy = 0.583982
Epoch 194
Loss = 5.4747e-01, PNorm = 27.6535, GNorm = 6.9915, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.565947
Epoch 195
Validation binary_cross_entropy = 0.546881
Epoch 196
Validation binary_cross_entropy = 0.534836
Epoch 197
Validation binary_cross_entropy = 0.531076
Epoch 198
Validation binary_cross_entropy = 0.526519
Epoch 199
Loss = 4.3260e-01, PNorm = 27.6572, GNorm = 4.6023, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.521174
Epoch 200
Validation binary_cross_entropy = 0.513274
Epoch 201
Validation binary_cross_entropy = 0.515000
Epoch 202
Validation binary_cross_entropy = 0.519027
Epoch 203
Validation binary_cross_entropy = 0.525515
Epoch 204
Loss = 3.9987e-01, PNorm = 27.6623, GNorm = 3.5975, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.519742
Epoch 205
Validation binary_cross_entropy = 0.517085
Epoch 206
Validation binary_cross_entropy = 0.532328
Epoch 207
Validation binary_cross_entropy = 0.554016
Epoch 208
Validation binary_cross_entropy = 0.572757
Epoch 209
Loss = 5.7086e-01, PNorm = 27.6674, GNorm = 5.9308, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.584088
Epoch 210
Validation binary_cross_entropy = 0.590334
Epoch 211
Validation binary_cross_entropy = 0.596893
Epoch 212
Validation binary_cross_entropy = 0.592248
Epoch 213
Validation binary_cross_entropy = 0.581859
Epoch 214
Loss = 4.9344e-01, PNorm = 27.6712, GNorm = 4.5566, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.575445
Epoch 215
Validation binary_cross_entropy = 0.560134
Epoch 216
Validation binary_cross_entropy = 0.540209
Epoch 217
Validation binary_cross_entropy = 0.521761
Epoch 218
Validation binary_cross_entropy = 0.516065
Epoch 219
Loss = 5.0887e-01, PNorm = 27.6750, GNorm = 5.3271, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.513254
Epoch 220
Validation binary_cross_entropy = 0.509338
Epoch 221
Validation binary_cross_entropy = 0.508797
Epoch 222
Validation binary_cross_entropy = 0.509549
Epoch 223
Validation binary_cross_entropy = 0.524665
Epoch 224
Loss = 5.1187e-01, PNorm = 27.6791, GNorm = 5.9951, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.538419
Epoch 225
Validation binary_cross_entropy = 0.556154
Epoch 226
Validation binary_cross_entropy = 0.573876
Epoch 227
Validation binary_cross_entropy = 0.584000
Epoch 228
Validation binary_cross_entropy = 0.588574
Epoch 229
Loss = 4.4053e-01, PNorm = 27.6829, GNorm = 4.7581, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.590398
Epoch 230
Validation binary_cross_entropy = 0.584016
Epoch 231
Validation binary_cross_entropy = 0.572469
Epoch 232
Validation binary_cross_entropy = 0.560080
Epoch 233
Validation binary_cross_entropy = 0.551094
Epoch 234
Loss = 3.6604e-01, PNorm = 27.6871, GNorm = 5.1696, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.547848
Epoch 235
Validation binary_cross_entropy = 0.545549
Epoch 236
Validation binary_cross_entropy = 0.542859
Epoch 237
Validation binary_cross_entropy = 0.542019
Epoch 238
Validation binary_cross_entropy = 0.545629
Epoch 239
Loss = 4.4006e-01, PNorm = 27.6912, GNorm = 5.5186, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.551979
Epoch 240
Validation binary_cross_entropy = 0.558935
Epoch 241
Validation binary_cross_entropy = 0.566035
Epoch 242
Validation binary_cross_entropy = 0.573954
Epoch 243
Validation binary_cross_entropy = 0.579946
Epoch 244
Loss = 4.6617e-01, PNorm = 27.6957, GNorm = 4.6734, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.581997
Epoch 245
Validation binary_cross_entropy = 0.589625
Epoch 246
Validation binary_cross_entropy = 0.592026
Epoch 247
Validation binary_cross_entropy = 0.587588
Epoch 248
Validation binary_cross_entropy = 0.581540
Epoch 249
Loss = 4.8960e-01, PNorm = 27.6996, GNorm = 3.9967, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.572351
Epoch 250
Validation binary_cross_entropy = 0.568859
Epoch 251
Validation binary_cross_entropy = 0.573566
Epoch 252
Validation binary_cross_entropy = 0.565996
Epoch 253
Validation binary_cross_entropy = 0.558626
Epoch 254
Loss = 4.1180e-01, PNorm = 27.7044, GNorm = 3.9610, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.551862
Epoch 255
Validation binary_cross_entropy = 0.544667
Epoch 256
Validation binary_cross_entropy = 0.544954
Epoch 257
Validation binary_cross_entropy = 0.555159
Epoch 258
Validation binary_cross_entropy = 0.568159
Epoch 259
Loss = 4.9022e-01, PNorm = 27.7097, GNorm = 5.8352, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.571817
Epoch 260
Validation binary_cross_entropy = 0.572822
Epoch 261
Validation binary_cross_entropy = 0.568400
Epoch 262
Validation binary_cross_entropy = 0.566852
Epoch 263
Validation binary_cross_entropy = 0.572198
Epoch 264
Loss = 3.6365e-01, PNorm = 27.7146, GNorm = 4.0559, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.580826
Epoch 265
Validation binary_cross_entropy = 0.588305
Epoch 266
Validation binary_cross_entropy = 0.583409
Epoch 267
Validation binary_cross_entropy = 0.577664
Epoch 268
Validation binary_cross_entropy = 0.573178
Epoch 269
Loss = 4.8491e-01, PNorm = 27.7180, GNorm = 6.1064, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.554280
Epoch 270
Validation binary_cross_entropy = 0.543079
Epoch 271
Validation binary_cross_entropy = 0.537009
Epoch 272
Validation binary_cross_entropy = 0.536274
Epoch 273
Validation binary_cross_entropy = 0.539097
Epoch 274
Loss = 4.6320e-01, PNorm = 27.7212, GNorm = 5.1969, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.544199
Epoch 275
Validation binary_cross_entropy = 0.549204
Epoch 276
Validation binary_cross_entropy = 0.552962
Epoch 277
Validation binary_cross_entropy = 0.555835
Epoch 278
Validation binary_cross_entropy = 0.560162
Epoch 279
Loss = 5.7386e-01, PNorm = 27.7248, GNorm = 5.6255, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.558919
Epoch 280
Validation binary_cross_entropy = 0.548726
Epoch 281
Validation binary_cross_entropy = 0.539495
Epoch 282
Validation binary_cross_entropy = 0.538259
Epoch 283
Validation binary_cross_entropy = 0.535325
Epoch 284
Loss = 4.7569e-01, PNorm = 27.7282, GNorm = 5.9616, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.533896
Epoch 285
Validation binary_cross_entropy = 0.532094
Epoch 286
Validation binary_cross_entropy = 0.533271
Epoch 287
Validation binary_cross_entropy = 0.533818
Epoch 288
Validation binary_cross_entropy = 0.537094
Epoch 289
Loss = 4.9874e-01, PNorm = 27.7313, GNorm = 4.3292, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.539226
Epoch 290
Validation binary_cross_entropy = 0.542171
Epoch 291
Validation binary_cross_entropy = 0.546008
Epoch 292
Validation binary_cross_entropy = 0.552389
Epoch 293
Validation binary_cross_entropy = 0.551986
Epoch 294
Loss = 4.6036e-01, PNorm = 27.7356, GNorm = 5.5717, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.547759
Epoch 295
Validation binary_cross_entropy = 0.547536
Epoch 296
Validation binary_cross_entropy = 0.545411
Epoch 297
Validation binary_cross_entropy = 0.544787
Epoch 298
Validation binary_cross_entropy = 0.537356
Epoch 299
Loss = 4.5559e-01, PNorm = 27.7400, GNorm = 3.9100, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.534472
Model 0 best validation binary_cross_entropy = 0.369693 on epoch 3
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.315227
Ensemble test binary_cross_entropy = 0.315227
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=300, out_features=400, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=400, out_features=400, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=400, out_features=1, bias=True)
  )
)
Number of parameters = 281,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.978999
Epoch 1
Validation binary_cross_entropy = 1.443949
Epoch 2
Validation binary_cross_entropy = 0.552435
Epoch 3
Validation binary_cross_entropy = 0.370025
Epoch 4
Loss = 3.1903e+00, PNorm = 27.3561, GNorm = 39.4348, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.351735
Epoch 5
Validation binary_cross_entropy = 0.887605
Epoch 6
Validation binary_cross_entropy = 1.841671
Epoch 7
Validation binary_cross_entropy = 0.869978
Epoch 8
Validation binary_cross_entropy = 0.399116
Epoch 9
Loss = 2.9244e+00, PNorm = 27.3720, GNorm = 47.2353, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.438921
Epoch 10
Validation binary_cross_entropy = 1.792952
Epoch 11
Validation binary_cross_entropy = 5.138522
Epoch 12
Validation binary_cross_entropy = 3.835007
Epoch 13
Validation binary_cross_entropy = 1.921251
Epoch 14
Loss = 2.4666e+00, PNorm = 27.3990, GNorm = 37.1302, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.748917
Epoch 15
Validation binary_cross_entropy = 0.768429
Epoch 16
Validation binary_cross_entropy = 1.466130
Epoch 17
Validation binary_cross_entropy = 2.283035
Epoch 18
Validation binary_cross_entropy = 2.189498
Epoch 19
Loss = 1.4541e+00, PNorm = 27.4235, GNorm = 22.1267, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 1.892563
Epoch 20
Validation binary_cross_entropy = 0.880356
Epoch 21
Validation binary_cross_entropy = 0.721493
Epoch 22
Validation binary_cross_entropy = 0.556075
Epoch 23
Validation binary_cross_entropy = 0.468210
Epoch 24
Loss = 1.9516e+00, PNorm = 27.4440, GNorm = 31.9578, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.553932
Epoch 25
Validation binary_cross_entropy = 1.054057
Epoch 26
Validation binary_cross_entropy = 1.255872
Epoch 27
Validation binary_cross_entropy = 0.855806
Epoch 28
Validation binary_cross_entropy = 0.571751
Epoch 29
Loss = 1.3650e+00, PNorm = 27.4608, GNorm = 20.9909, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.506414
Epoch 30
Validation binary_cross_entropy = 0.537768
Epoch 31
Validation binary_cross_entropy = 0.654437
Epoch 32
Validation binary_cross_entropy = 1.015183
Epoch 33
Validation binary_cross_entropy = 1.047531
Epoch 34
Loss = 1.3239e+00, PNorm = 27.4732, GNorm = 20.7503, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.785205
Epoch 35
Validation binary_cross_entropy = 0.499401
Epoch 36
Validation binary_cross_entropy = 0.406125
Epoch 37
Validation binary_cross_entropy = 0.488989
Epoch 38
Validation binary_cross_entropy = 0.685733
Epoch 39
Loss = 1.2437e+00, PNorm = 27.4841, GNorm = 14.0255, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.851774
Epoch 40
Validation binary_cross_entropy = 0.644870
Epoch 41
Validation binary_cross_entropy = 0.451721
Epoch 42
Validation binary_cross_entropy = 0.390267
Epoch 43
Validation binary_cross_entropy = 0.370065
Epoch 44
Loss = 1.2858e+00, PNorm = 27.4940, GNorm = 14.6868, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.417603
Epoch 45
Validation binary_cross_entropy = 0.522250
Epoch 46
Validation binary_cross_entropy = 0.598600
Epoch 47
Validation binary_cross_entropy = 0.529090
Epoch 48
Validation binary_cross_entropy = 0.452628
Epoch 49
Loss = 1.1388e+00, PNorm = 27.5034, GNorm = 16.7787, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.461356
Epoch 50
Validation binary_cross_entropy = 0.491278
Epoch 51
Validation binary_cross_entropy = 0.557859
Epoch 52
Validation binary_cross_entropy = 0.639663
Epoch 53
Validation binary_cross_entropy = 0.690435
Epoch 54
Loss = 8.3165e-01, PNorm = 27.5128, GNorm = 14.1429, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.694378
Epoch 55
Validation binary_cross_entropy = 0.571140
Epoch 56
Validation binary_cross_entropy = 0.565914
Epoch 57
Validation binary_cross_entropy = 0.608376
Epoch 58
Validation binary_cross_entropy = 0.683744
Epoch 59
Loss = 8.3977e-01, PNorm = 27.5203, GNorm = 13.7141, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.830108
Epoch 60
Validation binary_cross_entropy = 0.936934
Epoch 61
Validation binary_cross_entropy = 0.948713
Epoch 62
Validation binary_cross_entropy = 0.848251
Epoch 63
Validation binary_cross_entropy = 0.671053
Epoch 64
Loss = 1.1718e+00, PNorm = 27.5262, GNorm = 12.8886, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.588660
Epoch 65
Validation binary_cross_entropy = 0.573940
Epoch 66
Validation binary_cross_entropy = 0.612437
Epoch 67
Validation binary_cross_entropy = 0.648939
Epoch 68
Validation binary_cross_entropy = 0.692549
Epoch 69
Loss = 7.0517e-01, PNorm = 27.5321, GNorm = 11.0693, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.728071
Epoch 70
Validation binary_cross_entropy = 0.771764
Epoch 71
Validation binary_cross_entropy = 0.836641
Epoch 72
Validation binary_cross_entropy = 0.851088
Epoch 73
Validation binary_cross_entropy = 0.827603
Epoch 74
Loss = 7.9268e-01, PNorm = 27.5377, GNorm = 10.1201, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.699094
Epoch 75
Validation binary_cross_entropy = 0.582257
Epoch 76
Validation binary_cross_entropy = 0.550653
Epoch 77
Validation binary_cross_entropy = 0.552730
Epoch 78
Validation binary_cross_entropy = 0.579062
Epoch 79
Loss = 1.1068e+00, PNorm = 27.5435, GNorm = 8.8940, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.659206
Epoch 80
Validation binary_cross_entropy = 0.723386
Epoch 81
Validation binary_cross_entropy = 0.754332
Epoch 82
Validation binary_cross_entropy = 0.712195
Epoch 83
Validation binary_cross_entropy = 0.711084
Epoch 84
Loss = 1.0092e+00, PNorm = 27.5479, GNorm = 12.3526, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.699440
Epoch 85
Validation binary_cross_entropy = 0.674308
Epoch 86
Validation binary_cross_entropy = 0.616298
Epoch 87
Validation binary_cross_entropy = 0.596195
Epoch 88
Validation binary_cross_entropy = 0.582457
Epoch 89
Loss = 8.0961e-01, PNorm = 27.5499, GNorm = 10.8696, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.519993
Epoch 90
Validation binary_cross_entropy = 0.515255
Epoch 91
Validation binary_cross_entropy = 0.544857
Epoch 92
Validation binary_cross_entropy = 0.598119
Epoch 93
Validation binary_cross_entropy = 0.682377
Epoch 94
Loss = 7.5671e-01, PNorm = 27.5543, GNorm = 15.7218, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.741941
Epoch 95
Validation binary_cross_entropy = 0.729806
Epoch 96
Validation binary_cross_entropy = 0.655169
Epoch 97
Validation binary_cross_entropy = 0.595123
Epoch 98
Validation binary_cross_entropy = 0.568002
Epoch 99
Loss = 7.8062e-01, PNorm = 27.5591, GNorm = 11.1816, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.550010
Epoch 100
Validation binary_cross_entropy = 0.557760
Epoch 101
Validation binary_cross_entropy = 0.580245
Epoch 102
Validation binary_cross_entropy = 0.621831
Epoch 103
Validation binary_cross_entropy = 0.667434
Epoch 104
Loss = 5.8313e-01, PNorm = 27.5643, GNorm = 9.9058, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.681318
Epoch 105
Validation binary_cross_entropy = 0.687292
Epoch 106
Validation binary_cross_entropy = 0.674436
Epoch 107
Validation binary_cross_entropy = 0.624842
Epoch 108
Validation binary_cross_entropy = 0.586623
Epoch 109
Loss = 4.6443e-01, PNorm = 27.5690, GNorm = 5.0555, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.571081
Epoch 110
Validation binary_cross_entropy = 0.545411
Epoch 111
Validation binary_cross_entropy = 0.523614
Epoch 112
Validation binary_cross_entropy = 0.525497
Epoch 113
Validation binary_cross_entropy = 0.542782
Epoch 114
Loss = 7.4781e-01, PNorm = 27.5734, GNorm = 8.7162, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.575182
Epoch 115
Validation binary_cross_entropy = 0.618813
Epoch 116
Validation binary_cross_entropy = 0.654344
Epoch 117
Validation binary_cross_entropy = 0.673906
Epoch 118
Validation binary_cross_entropy = 0.681477
Epoch 119
Loss = 7.9265e-01, PNorm = 27.5771, GNorm = 11.4438, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.693186
Epoch 120
Validation binary_cross_entropy = 0.694585
Epoch 121
Validation binary_cross_entropy = 0.682660
Epoch 122
Validation binary_cross_entropy = 0.659491
Epoch 123
Validation binary_cross_entropy = 0.625153
Epoch 124
Loss = 7.2899e-01, PNorm = 27.5814, GNorm = 8.1482, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.593829
Epoch 125
Validation binary_cross_entropy = 0.574389
Epoch 126
Validation binary_cross_entropy = 0.564794
Epoch 127
Validation binary_cross_entropy = 0.581006
Epoch 128
Validation binary_cross_entropy = 0.610302
Epoch 129
Loss = 4.9588e-01, PNorm = 27.5868, GNorm = 5.4440, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.642889
Epoch 130
Validation binary_cross_entropy = 0.659208
Epoch 131
Validation binary_cross_entropy = 0.663104
Epoch 132
Validation binary_cross_entropy = 0.662046
Epoch 133
Validation binary_cross_entropy = 0.653959
Epoch 134
Loss = 6.0633e-01, PNorm = 27.5915, GNorm = 5.9137, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.651305
Epoch 135
Validation binary_cross_entropy = 0.629059
Epoch 136
Validation binary_cross_entropy = 0.609542
Epoch 137
Validation binary_cross_entropy = 0.608371
Epoch 138
Validation binary_cross_entropy = 0.621322
Epoch 139
Loss = 6.9482e-01, PNorm = 27.5953, GNorm = 7.8945, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.628619
Epoch 140
Validation binary_cross_entropy = 0.639215
Epoch 141
Validation binary_cross_entropy = 0.653070
Epoch 142
Validation binary_cross_entropy = 0.655258
Epoch 143
Validation binary_cross_entropy = 0.653432
Epoch 144
Loss = 4.9117e-01, PNorm = 27.5990, GNorm = 7.1722, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.649970
Epoch 145
Validation binary_cross_entropy = 0.648694
Epoch 146
Validation binary_cross_entropy = 0.658518
Epoch 147
Validation binary_cross_entropy = 0.658980
Epoch 148
Validation binary_cross_entropy = 0.637765
Epoch 149
Loss = 6.3233e-01, PNorm = 27.6037, GNorm = 7.6141, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.610383
Epoch 150
Validation binary_cross_entropy = 0.592996
Epoch 151
Validation binary_cross_entropy = 0.584601
Epoch 152
Validation binary_cross_entropy = 0.582516
Epoch 153
Validation binary_cross_entropy = 0.590543
Epoch 154
Loss = 5.9073e-01, PNorm = 27.6067, GNorm = 6.8673, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.605367
Epoch 155
Validation binary_cross_entropy = 0.627141
Epoch 156
Validation binary_cross_entropy = 0.639833
Epoch 157
Validation binary_cross_entropy = 0.661916
Epoch 158
Validation binary_cross_entropy = 0.672204
Epoch 159
Loss = 5.0652e-01, PNorm = 27.6091, GNorm = 6.1717, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.670057
Epoch 160
Validation binary_cross_entropy = 0.663732
Epoch 161
Validation binary_cross_entropy = 0.654597
Epoch 162
Validation binary_cross_entropy = 0.653322
Epoch 163
Validation binary_cross_entropy = 0.636306
Epoch 164
Loss = 5.2624e-01, PNorm = 27.6131, GNorm = 7.4841, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.622749
Epoch 165
Validation binary_cross_entropy = 0.602919
Epoch 166
Validation binary_cross_entropy = 0.581221
Epoch 167
Validation binary_cross_entropy = 0.574660
Epoch 168
Validation binary_cross_entropy = 0.578581
Epoch 169
Loss = 6.7991e-01, PNorm = 27.6162, GNorm = 6.0654, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.587142
Epoch 170
Validation binary_cross_entropy = 0.591429
Epoch 171
Validation binary_cross_entropy = 0.596369
Epoch 172
Validation binary_cross_entropy = 0.608422
Epoch 173
Validation binary_cross_entropy = 0.625653
Epoch 174
Loss = 5.6759e-01, PNorm = 27.6198, GNorm = 6.4088, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.627722
Epoch 175
Validation binary_cross_entropy = 0.627768
Epoch 176
Validation binary_cross_entropy = 0.620662
Epoch 177
Validation binary_cross_entropy = 0.609191
Epoch 178
Validation binary_cross_entropy = 0.600569
Epoch 179
Loss = 6.7640e-01, PNorm = 27.6238, GNorm = 8.1881, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.590322
Epoch 180
Validation binary_cross_entropy = 0.580938
Epoch 181
Validation binary_cross_entropy = 0.579157
Epoch 182
Validation binary_cross_entropy = 0.569380
Epoch 183
Validation binary_cross_entropy = 0.566397
Epoch 184
Loss = 5.1846e-01, PNorm = 27.6284, GNorm = 6.5730, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.567588
Epoch 185
Validation binary_cross_entropy = 0.562692
Epoch 186
Validation binary_cross_entropy = 0.562349
Epoch 187
Validation binary_cross_entropy = 0.568225
Epoch 188
Validation binary_cross_entropy = 0.566404
Epoch 189
Loss = 5.4745e-01, PNorm = 27.6326, GNorm = 6.9407, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.563202
Epoch 190
Validation binary_cross_entropy = 0.566359
Epoch 191
Validation binary_cross_entropy = 0.574010
Epoch 192
Validation binary_cross_entropy = 0.582024
Epoch 193
Validation binary_cross_entropy = 0.585596
Epoch 194
Loss = 5.2921e-01, PNorm = 27.6360, GNorm = 4.3927, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.592090
Epoch 195
Validation binary_cross_entropy = 0.604658
Epoch 196
Validation binary_cross_entropy = 0.612958
Epoch 197
Validation binary_cross_entropy = 0.614770
Epoch 198
Validation binary_cross_entropy = 0.615677
Epoch 199
Loss = 5.9443e-01, PNorm = 27.6391, GNorm = 5.9811, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.613395
Epoch 200
Validation binary_cross_entropy = 0.609824
Epoch 201
Validation binary_cross_entropy = 0.603711
Epoch 202
Validation binary_cross_entropy = 0.591300
Epoch 203
Validation binary_cross_entropy = 0.578995
Epoch 204
Loss = 4.7016e-01, PNorm = 27.6428, GNorm = 6.2947, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.567103
Epoch 205
Validation binary_cross_entropy = 0.558538
Epoch 206
Validation binary_cross_entropy = 0.550855
Epoch 207
Validation binary_cross_entropy = 0.550229
Epoch 208
Validation binary_cross_entropy = 0.553052
Epoch 209
Loss = 4.7845e-01, PNorm = 27.6467, GNorm = 3.9381, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.560247
Epoch 210
Validation binary_cross_entropy = 0.569906
Epoch 211
Validation binary_cross_entropy = 0.580711
Epoch 212
Validation binary_cross_entropy = 0.585176
Epoch 213
Validation binary_cross_entropy = 0.586543
Epoch 214
Loss = 5.4929e-01, PNorm = 27.6508, GNorm = 4.4665, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.592031
Epoch 215
Validation binary_cross_entropy = 0.593756
Epoch 216
Validation binary_cross_entropy = 0.585492
Epoch 217
Validation binary_cross_entropy = 0.577184
Epoch 218
Validation binary_cross_entropy = 0.577175
Epoch 219
Loss = 5.1863e-01, PNorm = 27.6551, GNorm = 6.5896, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.579521
Epoch 220
Validation binary_cross_entropy = 0.582787
Epoch 221
Validation binary_cross_entropy = 0.583189
Epoch 222
Validation binary_cross_entropy = 0.585313
Epoch 223
Validation binary_cross_entropy = 0.590277
Epoch 224
Loss = 5.9960e-01, PNorm = 27.6581, GNorm = 6.6502, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.594767
Epoch 225
Validation binary_cross_entropy = 0.602024
Epoch 226
Validation binary_cross_entropy = 0.607841
Epoch 227
Validation binary_cross_entropy = 0.611173
Epoch 228
Validation binary_cross_entropy = 0.604955
Epoch 229
Loss = 4.4691e-01, PNorm = 27.6610, GNorm = 4.3322, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.594208
Epoch 230
Validation binary_cross_entropy = 0.589378
Epoch 231
Validation binary_cross_entropy = 0.591234
Epoch 232
Validation binary_cross_entropy = 0.592420
Epoch 233
Validation binary_cross_entropy = 0.596252
Epoch 234
Loss = 4.5438e-01, PNorm = 27.6652, GNorm = 3.7552, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.604627
Epoch 235
Validation binary_cross_entropy = 0.612313
Epoch 236
Validation binary_cross_entropy = 0.616827
Epoch 237
Validation binary_cross_entropy = 0.619310
Epoch 238
Validation binary_cross_entropy = 0.626433
Epoch 239
Loss = 5.5436e-01, PNorm = 27.6703, GNorm = 5.5745, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.631370
Epoch 240
Validation binary_cross_entropy = 0.630862
Epoch 241
Validation binary_cross_entropy = 0.628301
Epoch 242
Validation binary_cross_entropy = 0.622241
Epoch 243
Validation binary_cross_entropy = 0.613904
Epoch 244
Loss = 4.0476e-01, PNorm = 27.6752, GNorm = 5.2775, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.606385
Epoch 245
Validation binary_cross_entropy = 0.605814
Epoch 246
Validation binary_cross_entropy = 0.609282
Epoch 247
Validation binary_cross_entropy = 0.616393
Epoch 248
Validation binary_cross_entropy = 0.621271
Epoch 249
Loss = 4.6632e-01, PNorm = 27.6792, GNorm = 5.1258, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.631587
Epoch 250
Validation binary_cross_entropy = 0.643408
Epoch 251
Validation binary_cross_entropy = 0.649144
Epoch 252
Validation binary_cross_entropy = 0.652852
Epoch 253
Validation binary_cross_entropy = 0.656594
Epoch 254
Loss = 6.9133e-01, PNorm = 27.6818, GNorm = 6.4394, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.660991
Epoch 255
Validation binary_cross_entropy = 0.659626
Epoch 256
Validation binary_cross_entropy = 0.650738
Epoch 257
Validation binary_cross_entropy = 0.648385
Epoch 258
Validation binary_cross_entropy = 0.634791
Epoch 259
Loss = 4.7166e-01, PNorm = 27.6857, GNorm = 6.6466, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.630945
Epoch 260
Validation binary_cross_entropy = 0.632762
Epoch 261
Validation binary_cross_entropy = 0.630471
Epoch 262
Validation binary_cross_entropy = 0.630127
Epoch 263
Validation binary_cross_entropy = 0.626257
Epoch 264
Loss = 4.6132e-01, PNorm = 27.6895, GNorm = 4.3179, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.627570
Epoch 265
Validation binary_cross_entropy = 0.628659
Epoch 266
Validation binary_cross_entropy = 0.623704
Epoch 267
Validation binary_cross_entropy = 0.621252
Epoch 268
Validation binary_cross_entropy = 0.627989
Epoch 269
Loss = 4.7808e-01, PNorm = 27.6935, GNorm = 4.9845, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.635453
Epoch 270
Validation binary_cross_entropy = 0.644455
Epoch 271
Validation binary_cross_entropy = 0.651165
Epoch 272
Validation binary_cross_entropy = 0.656583
Epoch 273
Validation binary_cross_entropy = 0.657844
Epoch 274
Loss = 4.8747e-01, PNorm = 27.6983, GNorm = 5.1409, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.656994
Epoch 275
Validation binary_cross_entropy = 0.651125
Epoch 276
Validation binary_cross_entropy = 0.649325
Epoch 277
Validation binary_cross_entropy = 0.643040
Epoch 278
Validation binary_cross_entropy = 0.637414
Epoch 279
Loss = 4.2003e-01, PNorm = 27.7025, GNorm = 3.6279, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.632264
Epoch 280
Validation binary_cross_entropy = 0.624575
Epoch 281
Validation binary_cross_entropy = 0.615527
Epoch 282
Validation binary_cross_entropy = 0.613052
Epoch 283
Validation binary_cross_entropy = 0.617945
Epoch 284
Loss = 4.6214e-01, PNorm = 27.7064, GNorm = 3.3016, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.620996
Epoch 285
Validation binary_cross_entropy = 0.613311
Epoch 286
Validation binary_cross_entropy = 0.605996
Epoch 287
Validation binary_cross_entropy = 0.601600
Epoch 288
Validation binary_cross_entropy = 0.598907
Epoch 289
Loss = 3.6755e-01, PNorm = 27.7100, GNorm = 4.2780, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.605095
Epoch 290
Validation binary_cross_entropy = 0.620377
Epoch 291
Validation binary_cross_entropy = 0.638128
Epoch 292
Validation binary_cross_entropy = 0.650267
Epoch 293
Validation binary_cross_entropy = 0.666867
Epoch 294
Loss = 3.9648e-01, PNorm = 27.7141, GNorm = 5.3447, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.681318
Epoch 295
Validation binary_cross_entropy = 0.685575
Epoch 296
Validation binary_cross_entropy = 0.680453
Epoch 297
Validation binary_cross_entropy = 0.668817
Epoch 298
Validation binary_cross_entropy = 0.655462
Epoch 299
Loss = 4.2499e-01, PNorm = 27.7189, GNorm = 5.3261, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.646554
Model 0 best validation binary_cross_entropy = 0.351735 on epoch 4
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.313781
Ensemble test binary_cross_entropy = 0.313781
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.286525
	Seed 1 ==> test binary_cross_entropy = 0.363082
	Seed 2 ==> test binary_cross_entropy = 0.272460
	Seed 3 ==> test binary_cross_entropy = 0.226135
	Seed 4 ==> test binary_cross_entropy = 0.298599
	Seed 5 ==> test binary_cross_entropy = 0.328809
	Seed 6 ==> test binary_cross_entropy = 0.194977
	Seed 7 ==> test binary_cross_entropy = 0.223320
	Seed 8 ==> test binary_cross_entropy = 0.315227
	Seed 9 ==> test binary_cross_entropy = 0.313781
Overall test binary_cross_entropy = 0.282292 +/- 0.050401
Elapsed time = 0:03:23
